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CHAPTER  I 


PROBLEM  DEFINITION 
Statement  of  Problem 

The  Department  of  Defense  spends  on  the  order  of 
$30  billion  annually  for  maintenance,  more  than  half  of 
which  is  for  personnel.  Since  this  expenditure  accounts 
for  20  to  25  percent  of  the  total  defense  budget,  even  a 
small  reduction  in  this  area  would  result  in  substantial 
savings  (33:1).  Much  emphasis  has  been  placed  on  improving 
maintenance  in  the  equipment  area  by  stressing  the  impor- 
tance of  reliability  and  maintainability  as  design  criteria 
and  by  developing  built-in  test  equipment.  However,  little 
attention  has  been  given  to  the  "people"  problem  in  main- 
tenance. 

Research  dealing  with  the  "people"  problem  in  main- 
tenance has  been  primarily  concerned  with  improving  the 
information  transfer  to  the  technician  on  the  job.  Past 
and  current  research  projects  on  maintenance  aids  indicate 
that  faster  and  better  maintenance  can  be  accomplished  if 
technicians  use  job  performance  aids  (JPAs)  (33:1).  The 
use  of  JPAs  offers  the  opportunity  for  reducing  training 
costs  by  enabling  less  skilled  personnel,  and  perhaps  fewer 
of  them,  to  carry  out  maintenance  tasks. 


1 


Although  these  research  findings  are  important, 
they  do  not  completely  solve  the  problem  of  sustaining  main- 
tenance productivity  with  fewer  personnel  and  at  less  cost. 
There  exists  a further  potential  for  savings  through 
improved  personnel  selection  procedures.  The  current  prac- 
tice is  to  assign  personnel  to  schools  and  subsequent  career 
fields  on  the  basis  of  aptitude  scores  (11:3).  While  apti- 
tude scores  are  a good  indicator  of  how  well  an  individual 
will  perform  in  the  school,  they  do  not  consistently  indi- 
cate how  well  he  will  perform  on  the  job  (20) . The  identi- 
fication of  differential  predictors  of  success  other  than 
aptitude  scores  could  lead  to  expanded  training  and  oper- 
ating economies  through  training  only  those  individual 
possessing  the  highest  potential  for  performance  in  a par- 
ticular type  of  job. 

Justification 

With  the  advent  of  a volunteer  force,  rising  man- 
power costs,  and  tighter  defense  budgets,  military  manpower 
has  become  one  of  the  most  important  issues  in  defense 
planning  and  budgeting.  Personnel  costs  increased  by  100 
percent  from  1962  to  1972.  In  1962  these  costs  comprised 
42  percent  of  the  total  defense  budget;  in  1972  they  were 
55  percent  of  the  total  defense  budget  (24:2).  The  DOD 
must  find  methods  of  increasing  the  productivity  of  its 
personnel  in  order  to  offset  cuts  in  both  manpower  and 
budget . 


In  a Rand  study  conducted  in  1972,  several  recom- 
mendations were  made  for  research  concerning  reduction  of 
personnel  costs  in  the  maintenance  area  (24:10-44).  One 
suggestion  was  that  performance  testing  should  be  under- 
taken for  estimating  production  and  the  factors  influencing 
it.  It  was  suggested  that  the  proposed  study  should  begin 
with  electronics  maintenance  since  the  potential  for 
savings  is  greatest  in  this  field. 

Electronics  maintenance  is  a costly  activity.  The 
Defense  Science  Board  estimated  that  electronics  mainte- 
nance consumed  25  percent  of  the  DOD  FY  1974  operations  and 
maintenance  budget,  or  about  $5.6  billion  (24:3).  Military 
pay  and  allowances  and  other  personnel  costs  for  personnel 
in  the  electronic  maintenance  area  are  about  $10,000  per 
man  per  year.  Training  costs  are  also  high.  Formal  train- 
ing schools,  which  consist  of  an  average  of  30  weeks,  cost 
approximately  $10,000  per  graduate.  Given  an  annual  turn- 
over rate  of  25  percent,  the  total  personnel  costs  in 
electronics  maintenance  is  about  $12,500  per  man  per  year. 
The  total  annual  costs  of  all  military  personnel  in  the 
electronics  field  is  approximately  $3.0  billion  (24:4). 

Past  studies  have  indicated  that  the  use  of  JPAs 
for  routine  maintenance  tasks  can  reduce  required  training 
time  and  make  an  individual  productive  almost  immediately. 
The  use  of  JPAs  in  electronics  maintenance  could  possibly 
result  in  savings  of  $10,000  per  trainee.  For  the  entire 


3 


DOD,  where  annual  training  output  in  electronics  mainte- 
nance is  approximately  40,000  men,  this  is  a potential 
savings  of  $400  million  (24:32). 

Improving  troubleshooting  procedures  has  also  been 
recognized  as  a method  of  increasing  productivity  and 
reducing  costs.  Thus,  a second  recommendation  of  the  Rand 
study  was  that  an  expanded  research  program  with  major 
emphasis  on  JPAs  as  a troubleshooting  aid  should  be  under- 
taken. It  was  suggested  that  this  expanded  research  should 
include  a study  of  which  troubleshooting  aids  are  the  most 
useful  to  the  technician  (24:39).  The  Rand  recommendation 
specifically  identified  troubleshooting  aids  because  the 
development  of  such  aids  is  inherently  more  difficult  than 
the  development  of  aids  for  routine  maintenance.  Trouble- 
shooting aids  must  lead  the  technician  from  initial  symptoms 
through  a series  of  branching  operations  to  isolate  the 
fault  (24:28). 

Based  on  these  recommendations,  the  Air  Force  Human 
Resources  Laboratory  (AFHRL)  recently  conducted  a research 
project  which  compared  the  performance  of  electronics  tech- 
nicians troubleshooting  with  three  different  types  of  aids. 
The  project  evaluated  sample  groups  composed  of  technicians 
at  three  different  experience  levels.  Like  the  research  in 
other  maintenance  areas,  the  AFHRL  project  concluded  that 
better  performance  could  be  achieved  by  the  use  of  the  more 
proceduralized  job  performance  aid  (29:45).  Since  the  use 
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of  JPAs  allows  the  use  of  less  skilled  personnel  and 
reduces  training  time,  the  AFHRL  findings  would  seem  to 
indicate  a great  potential  for  cost  reductions.  However, 
the  introduction  of  JPAs  does  not  automatically  result  in 
savings  (24:19). 

While  past  research  has  shown  the  potential  for 
cost  reductions  in  the  use  of  JPAs,  actual  savings  require, 
among  other  things,  reductions  in  training  and  manning 
levels  (24:19).  However,  training  and  manning  levels  can 
not  be  reduced  at  the  expense  of  maintenance  productivity. 
During  the  AFHRL  project,  casual  observations  indicated 
that  the  same  individuals  were  the  top  performers  in  each 
sample  group  regardless  of  which  type  of  troubleshooting 
aid  was  being  tested  (39) . These  observations  imply  that 
there  are  individuals  who  possess  a native  ability  for 
troubleshooting.  Since  the  problem  facing  DOD  is  to  main- 
tain or  increase  productivity  with  fewer  personnel,  it 
would  be  advantageous  to  be  able  to  identify  individuals 
possessing  the  prerequisite  abilities  prior  to  selection 
for  training  in  complex  electronics  maintenance  fields. 

To  identify  these  individuals,  it  is  necessary  to  find  a 
variable,  or  variables,  which  indicate  inherent  trouble- 
shooting abilities  and  can  be  used  as  predictors  of  job 
performance  and  productivity.  The  ability  to  predict  per- 
formance would  have  a great  impact  on  personnel  selection 
and  training  procedures. 


5 


Research  Objectives 

The  objective  of  this  research  is  to  confirm  that 
individuals  possessing  native  abilities  for  troubleshooting 
tasks  actually  exist.  Once  this  confirmation  is  accom- 
plished, a second  objective  is  to  determine  whether  per- 
sonality assessment  techniques  can  be  used  to  identify 
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troubleshooting  capability  potential  prior  to  actual  train- 
ing and  career  field  assignments.  A third  objective  of 
this  research  is  to  confirm  the  belief  that  aptitude  mea- 
sures are  an  accurate  indicator  of  actual  troubleshooting 
performance. 

Study  Approach 

To  meet  the  objective  of  confirming  that  individ- 
uals possessing  inherent  abilities  for  troubleshooting  do 
exist,  this  research  used  the  data  collected  on  trouble- 
shooting performance  during  the  AFHRL  project.  The  approach 
taken  was  to  use  statistical  analysis  techniques  to  determine 
if  the  same  individuals  were  the  high  performers  using  all 
three  types  of  troubleshooting  aids.  This  approach  was  not 
an  attempt  to  differentiate  "good"  performance  from  "bad" 
performance.  Although  an  individual  might  attain  a higher 
performance  rating  with  one  type  of  aid  than  with  another, 
if  it  could  be  shown  that  he  is  a consistently  high  per- 
former relative  to  other  troubleshooters,  regardless  of 
the  type  aid  used,  then  it  could  be  inferred  that  he  pos- 
sesses a higher  degree  of  native  ability. 
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In  addition  to  collecting  data  on  performance,  the 
AFHRL  project  also  administered  personality  tests  to  each 


subject.  Once  the  first  objective  was  achieved,  statistical 
analysis  of  the  performance  ratings  and  personality  test 
scores  was  conducted  to  determine  if  any  of  the  personality 
traits  measured  could  be  related  to  troubleshooting  per- 
formance and  therefore  could  be  used  as  predictors  of  suc- 
cess in  troubleshooting  tasks.  Prior  to  the  analysis  of 
personality  test  scores,  however,  it  was  necessary  to  deter- 
mine if  there  was  a significant  difference  in  the  perform- 
ance of  the  technicians  at  the  three  different  experience 
levels  since  a difference  in  performance  between  the  three 
groups  would  require  that  the  techniques  used  in  the  analysis 
of  performance  ratings  and  personality  test  scores  be  applied 
to  each  group  separately. 

The  third  research  objective  dealt  with  aptitude 
measures.  The  Airman  Qualifying  Examination  (AQE)  scores 
were  available  for  each  of  the  subjects  evaluated  in  the 
AFHRL  project.  Statistical  analysis  of  the  AQE  scores  and 
the  performance  ratings  was  performed  to  determine  if  a 
relationship  exists  between  the  aptitude  measures  and 
troubleshooting  performance. 

Definition  of  Terms 

Job  Performance  Aid  (JPA)--a  type  of  maintenance 


aids  in  that  it  attempts  to  lay  out  in  logical  sequence 
the  tasks  that  should  be  performed.  It  may  be  fully  pro- 
ceduralized  and  provide  step-by-step  directions  or  it  may 
provide  a logical  picture  of  the  system  to  assist  the  tech- 
nician in  deciding  what  steps  to  take  (24:29). 

Maintenance  Aid — stored  information,  either  in 
devices  or  documents,  available  at  the  job  site  to  assist 
the  technician  in  performing  his  task  (24:29). 

Maintenance  Dependency  Charts  (MDC) — a type  of 
decision  aid  which  helps  the  technician  formulate  a strat- 
egy for  isolating  faults.  The  MDC  is  a device  which  helps 
in  understanding  the  interrelationships  of  a complex  elec- 
tronics system  (24:29). 

Troubleshooting — the  process  of  diagnosing, 
locating,  and  repairing  a malfunction  and/or  failure  in  a 
system  or  subsystem  (7:1). 
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CHAPTER  II 


THEORIES  AND  CONCEPTS 
Personality  Assessment 

Individual  differences  and  their  effect  on  indi- 
vidual behavior  has  long  been  recognized.  Likewise,  man 
has  long  been  interested  in  methods  of  assessing  these 
differences  and  using  this  assessment  to  predict  behavior. 
Early  attempts  at  personality  assessment  included  such 
methods  as  astrology,  palmistry,  and  phrenology  (21:2). 
These  methods  have  not  survived  scientific  investigations. 

The  scientific  measurement  of  personality  began 
only  a few  decades  ago  (9:4).  It  has  its  roots  in  the 
study  of  individual  differences  through  psychological  mea- 
surement. This  concept  was  given  impetus  by  Darwin's 
theory  of  evolution  (21:6).  In  order  to  study  the  effects 
of  human  generics,  it  was  first  necessary  to  identify  the 
individual  traits  which  were  related  to  adaptability  and 
survival. 

The  history  of  modern  personality  measurement  dates 
from  about  World  War  I (9:5).  During  this  time  two  differ- 
ent tests  were  developed.  The  first  was  Woodsworth's 
Personal  Data  Sheet  (21:9).  This  was  a questionnaire  type 
test  used  to  predict  the  ability  of  soldiers  to  adjust  to 
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the  strains  and  stresses  of  military  life.  During  this 
same  period,  Rorschach's  "Ink  Blot"  test  was  developed, 
which  used  an  individual's  perception  of  ink  blots  as  an 
indicator  of  personality  differences  (21:10).  These 
methods  have  been  expanded  upon  and  many  new  methods  have 
been  developed  in  recent  years. 

Personality  Defined 

Personality  can  be  defined  in  many  ways.  The 
definition  which  best  fits  the  purpose  of  this  research  is 
"the  nonintellective  attitudes  that  play  roles  in  job 
selection,  classification,  and  performance  [9:16]."  This 
definition  describes  those  attributes  which  most  interest 
the  personnel  manager.  The  personality  test  is  the  instru- 
ment used  to  measure  these  attributes.  The  objective  of  a 
personality  test  used  in  this  context  is  to  determine 
whether  the  person  being  tested  falls  in  the  class  of  indi- 
viduals who  are  most  likely  to  succeed  on  a particular  type 
of  job  or  who  will  benefit  most  from  a certain  training 
program  (9:18).  The  test  is  uniformly  applied  which  gives 
it  some  advantages  over  more  subjective  methods  of  per- 
sonality assessment  (3:41).  Many  personnel  managers  feel 
that  personality  is  the  most  important  determinant  of 
success  or  failure,  particularly  in  jobs  requiring  the 
exercise  of  supervision  (36:165). 
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Personality  Tests 

The  term  "personality  test"  may  refer  to  a variety 
of  psychological  measuring  instruments  (17:23).  The  two 
most  often  used  personality  measuring  devices  are  paper- 
and-pencil  inventories  and  projective  techniques  (36:165). 

Paper-and-pencil  inventories  are  composed  of  a 
series  of  questions  or  statements.  There  are  no  right  or 
wrong  answers.  The  person  taking  the  test  is  asked  to 
tell  how  he  feels  or  would  react  in  a given  situation 
(40:189).  He  may  be  asked  to  respond  by  answering  "yes" 
or  "no"  or  "sometimes,"  "always,"  or  "never;"  or  he  may 
be  given  a multiple  list  of  answers  from  which  to  choose 
(36:165).  The  responses  are  scored  for  particular  traits 
or  attributes.  The  responses  which  are  being  scored  have 
usually  been  identified  by  statistical  analysis  or  by 
subjective  means  as  being  responses  that  are  characteristic 
of  that  trait  or  attribute  (40:190). 

The  projective  techniques  test  confronts  the  sub- 
ject with  several  items  such  as  ink  blots  or  pictures.  It 
must  be  administered  individually  rather  than  in  groups. 

The  subject  is  encouraged  to  respond  freely,  telling  what 
he  sees  in  the  blot  or  making  up  a story  about  a picture. 
His  responses  are  presumed  to  be  projections  of  his 
thoughts,  wishes,  desires,  and  needs  (36:166).  A trained 
test  administrator  interprets  the  responses  and  makes  an 
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assessment  of  the  subject,  which  is  usually  in  qualitative 
rather  than  quantitative  terms  (40:190). 

Personality  type  tests  can  be  structured  to  measure 
different  facets  of  personality.  Different  types  of  per- 
sonality tests  can  be  grouped  according  to  the  particular 
attribute  that  they  are  attempting  to  measure.  These  major 
groupings  are  interest,  preference,  behavior,  and  attitude 
(17:22) . 

Interest  tests,  as  the  name  implies,  are  designed 
to  indicate  things  the  subject  might  be  interested  in. 

This  type  personality  test  is  one  of  the  most  familiar. 

The  test  consists  of  a variety  of  items,  each  of  which 
calls  for  a response  indicating  the  subject's  interest  in 
the  activity  or  subject  specified  by  that  item  (17:22). 

Preference  tests  are  characterized  by  the  format 
in  which  they  are  presented.  The  test  items  consist  of 
things  which  a person  might  say  about  himself  or  about 
others.  The  subject  indicates  his  preference  by  choosing 
the  response  which  most  closely  describes  his  feelings 
(17:22). 

Behavioral  tests  are  composed  of  items  which  are 
statements  of  specific  observable  behavior,  such  as  "going 
to  a movie  [17:23]."  This  test  is  based  on  the  assumption 
that  what  a person  actually  does  is  a more  accurate  mea- 
surement of  personality  than  what  his  interests  or  prefer- 
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The  most  distinct  of  the  personality  tests  is  the 
attitude  measurement  inventory.  The  attitude  statements 
used  in  this  test  are  stimulus  elements  to  which  the  sub- 
ject responds  in  terms  of  his  likes,  dislikes,  interest, 
feelings,  etc.,  just  as  he  does  in  other  types  of  tests. 

The  test  is  usually  made  up  of  a series  of  items  that  refer 
to  some  specific  organization  such  as  the  church  or  a 
social  group,  or  to  specific  issues,  social  institutions 
or  any  other  subjects  which  can  be  defined  (17:24). 

Another  application  of  attitude  tests  is  the  evalu- 
ation of  one  individual  by  another.  This  form  of  the  test 
would  consist  of  items  that  describe  the  performance  of  an 
employee  on  the  job  or  other  activity  (17:24).  In  this 
form,  the  test  would  be  an  attitude  questionnaire  used  by 
a supervisor  to  indicate  his  attitude  towards  a subordi- 
nate's performance. 

For  any  test  of  individual  differences  to  be  effec- 
tive it  must  have  adequate  measurements.  There  are  many 
factors  which  determine  the  adequacy  of  the  measurements; 
however,  a measurement  can  generally  be  described  as  ade- 
quate if  it  meets  three  criteria  (9:250). 

The  first  criterion  is  that  the  responses  from 
which  the  indices  of  measurement  are  obtained  must  be 
determined  as  completely  as  possible  by  one  important  per- 
sonality attribute  (9:250).  In  other  words,  there  must  be 
a direct  relationship  between  the  subject's  response  to  a 


particular  item  and  some  characteristic  of  his  personality. 

This  criterion  is  simply  asking  whether  the  test  is  mea- 
suring some  important  attribute  well. 

A second  criterion  is  closely  related  to  the  first. 

Besides  being  able  to  adequately  measure  a particular 
attribute,  the  test  items  must  indicate  the  degree  of  the 
attribute  characterizing  the  subject  at  that  period  of  his 
life  (9:251).  This  criterion  concerns  measuring  a trait 
rather  than  the  state  in  which  the  subject  happens  to  be. 

The  state  indicates  how  the  subject  feels  under  particular 
conditions.  The  personality  test  is  concerned  more  with 
measuring  under lying  determinants  of  behavior.  Trait  mea- 
surements are  an  average  over  most  situations,  or  a 
generalization  of  how  a subject  usually  feels  (9:180). 

Economy  and  convenience  require  that  the  measurement  be 
administered  at  one  point  in  time.  However,  if  this 
criterion  is  met,  a measurement  index  will  be  obtained 
which  will  closely  approximate  the  mean  index  obtained  by 
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administering  the  test  a number  of  times.  To  meet  ’ 
this  criterion,  the  setting,  the  task,  and  the  test  items 
must  have  a clear  psychological  meaning  for  the  subject.  . j 

When  these  elements  of  the  testing  environment  have  suffi- 
ciently strong  impact,  they  determine  the  subject's 
responses  (9:251).  Influences  from  temporary  states  or 
other  variables  are  elminated. 


The  third  criterion  is  that  the  indices  of  measure- 
ment must  differentiate  dependably  among  subjects  (9:251). 
The  purpose  of  the  personality  test  is  to  measure  differ- 
ences among  subjects  or  particular  traits.  It  is  assumed 
that  such  differences  exist.  The  procedures  used  to  mea- 
sure this  difference  must  produce  a range  of  scores  over 
different  subjects. 

The  personality  test  is  constructed  to  validly 
measure  some  individual  trait  difference.  These  three 
criteria  overlap  to  some  extent  in  that  they  are  all 
related  to  that  objective.  But  taken  separately,  they 
each  indicate  a particular  area  which  might  affect  the 
validity  of  the  test. 

Test  Validity 

The  most  important  question  to  be  asked  about  a 
psychological  test  of  any  kind  concerns  its  validity. 
Validity  is  the  degree  to  which  the  test  actually  measures 
what  it  is  supposed  to  measure  (1:107).  Since  there  are 
different  tests  attempting  to  measure  different  personality 
attributes,  there  are  different  methods  of  validation.  A 
method  which  would  be  appropriate  for  demonstrating  the 
validity  of  one  test  would  not  necessarily  be  appropriate 
for  another  (21:128).  There  are  some  general  methods  of 
validation  which  are  recognized  as  being  appropriate.  Each 
method  applies  to  particular  objectives  for  a test  (40:128). 


Content  validity  is  evaluated  by  showing  how  well 


the  content  of  the  test  samples  the  subject  matter  or  kinds 
of  situations  that  the  test  is  supposed  to  measure  (40:128). 
This  type  of  validity  is  not  based  on  criteria.  No  attempt 
is  made  to  gather  data  or  information  to  form  a statisti- 
cal basis  for  prediction.  This  method  is  based  on  the 
judgement  of  experts  in  a particular  field  (3:43).  Although 
content  validity  applies  particularly  to  achievement  and 
aptitude  tests,  personality  tests  also  require  its  use  to 
some  extent  (21:128).  For  example,  a test  designed  to 
measure  leadership  qualities  might  be  composed  of  items 
which  require  responses  that  would  indicate  the  subject's 
behavior  in  situations  requiring  leadership  abilities.  If 
these  test  situations  are  representative  of  leadership 
situations  in  general,  then  the  test  can  be  said  to  display 
a content  validity  (21:129). 

A problem  in  using  the  content  validity  method  is 
the  possibility  that  the  test  is  based  on  "face  validity" 
(3:43).  That  is,  the  contents  appear  to  be  meaningful  but 
are  not  truly  significant.  This  is  particularly  an  argu- 
ment against  paper-and-pencil  personality  tests  (21:129). 
Therefore,  content  validity,  although  an  important  require- 
ment, is  not  sufficient  in  itself  for  determining  the  use- 
fulness of  a test.  Other  methods  of  validity  determination 
also  must  be  used  (3:48). 
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Personality  assessment  is  most  often  concerned 


with  predictive  validity  (21:130).  Predictive  validity  is 
evaluated  by  showing  how  well  predictions  made  by  the  test 
are  confirmed  at  some  future  time  (40:128).  It  is  not 
necessarily  concerned  with  predicting  the  future  but  rather 
with  predicting  relationships  (5:96-97).  Predictive  valid- 
ity refers  to  "the  accuracy  with  which  we  can  make  guesses 
about  one  characteristic  of  an  individual  from  another 
characteristic  [14:338]."  The  test  measure  is  called  the 
predictor  and  the  characteristic  being  guessed  at  is 
called  the  criterion.  The  predictive  validity  is  deter- 
mined by  a numerical  index.  This  index  is  given  by  a cor- 
relation between  the  predictor  and  the  criterion  (21:130). 

Another  validity  measurement  is  concurrent  validity. 
Concurrent  validity  is  evaluated  by  showing  how  much  the 
scores  on  a test  are  related  to  some  present  criterion  of 
performance  or  status  (40:128) . Rather  than  establishing 
a correlation  between  present  test  data  and  criterion  data 
available  at  some  later  time  (as  in  predictive  validity) , 
concurrent  validity  establishes  a correlation  between  test 
data  and  concurrently  available  criterion  data  (23:77). 

The  object  is  to  take  some  known  criterion,  such  as  job 
performance,  and  relate  it  to  some  test  score. 

Caution  must  be  used  in  determining  the  validity  of 
a test  with  concurrent  information.  A high  correlation  may 
mean  that  experience  is  being  tested  rather  than  the 
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desired  characteristic  (3:43).  When  this  happens,  the 
experience  factor  is  known  as  a contaminant.  Such  contam- 
ination results  when  the  influence  of  any  extraneous  fac- 
tors are  reflected  in  the  measurement  (36:102).  Statis- 
tical means  should  be  used  to  remove  any  such  contaminants 
from  the  tests  (3:43). 

The  last  of  the  general  methods  of  determining 
validity  is  construct  validity.  A construct  can  be  defined 
as  the  trait  which  the  test  is  attempting  to  measure  (23: 
90) . Construct  validity  is  evaluated  by  determining  what 
trait  or  attributes  a test  measures  (40:128).  This  method 
is  sometimes  used  when  a test  lacks  appropriate  criteria 
against  which  it  can  be  evaluated  (3:44).  Frequently,  con- 
struct validity  is  determined  by  correlating  one  test  with 
another  which  has  previously  been  used  to  measure  the  same 
attribute.  Another  method  would  be  to  use  factor  analysis, 
a statistical  process  which  can  identify  the  extent  to 
which  various  tests  measure  the  same  attribute  (40:128). 

Any  of  these  methods  of  determining  validity  could 
be  used  for  different  personality  type  tests.  However,  in 
keeping  with  the  earlier  definition  of  personality,  the  two 
most  often  used  methods  would  be  predictive  and  concurrent 
validity.  In  both  of  these  concepts  of  validity,  a 
criterion  of  job  performance  is  usually  involved  (40:128). 
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Test  Reliability 


The  reliability  of  a test  is  also  important  in 
determining  its  usefulness.  As  used  in  the  measurement  of 
individual  difference,  reliability  means  consistency  (1:67). 
The  reliability  of  a test  is  the  degree  to  which  the  test 
measures  consistently  the  attribute  it  is  supposed  to  mea- 
sure (40:129).  There  are  several  methods  of  determining 
test  reliability. 

The  test-retest  method  determines  the  reliability 
of  a test  over  time.  A reliable  test  is  one  in  which  an 
individual  will  obtain  about  the  same  score  on  successive 
testings  (36:141).  Retest  reliability  is  probably  the  most 
realistic  and  useful  method  from  a practical  point  of 
view  (17:41).  However,  if  memory  or  practice  from  the  first 
test  can  affect  scores  on  subsequent  tests,  this  method 
should  not  be  used  (40:129). 

A second  method  of  determining  test  reliability  is 
comparable  form  or  alternate-forms  reliability.  This 
method  uses  two  or  more  different  tests  that  are  supposed 
to  measure  the  same  attribute  (17:39).  A correlation  is 
then  established  between  the  scores  obtained  on  each  of 
these  tests  to  determine  the  degree  of  reliability  (40:129). 

A common  criticism  of  both  the  test-retest  and 
comparable  forms  method  is  that  they  require  two  separate 
testing  sessions.  A third  method  of  determining  relia- 
bility, the  split-halves  method,  requires  only  a single 
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test  administration.  The  total  test  is  divided  into  halves. 


Each  half  of  the  test  is  a miniature  representative  of  the 
entire  test  (36:143).  Each  half  is  scored  separately  and 
these  two  scores  are  then  correlated  to  determine  test 
reliability  (40:129). 

Test  reliability  and  validity  must  be  considered 
together.  A test  may  have  high  reliability  and  still  be 
worthless  in  certain  situations  (40:130).  However,  reli- 
ability is  important  because  it  limits  validity  (9:171). 
Theoretically,  a test's  maximum  potential  validity  is 
given  by  the  square  root  of  its  reliability.  For  example, 
if  a test's  reliability  coefficient  is  determined  to  be 
0.36,  its  validity  coefficient  can  not  exceed  0.60  (36:146). 

Since  maximum  validity  is  determined  by  reliability, 
the  choice  of  tests  to  be  used  depends  upon  the  coefficient 
of  validity  required.  This  requirement  is  determined  by 
what  use  is  to  be  made  of  the  test.  Personality  tests  are 
usually  used  for  one  of  two  objectives:  to  make  an  analysis 
of  each  individual  subject  tested  or  to  select  from  a large 
group  of  subjects  those  who  will  surpass  the  rest  in  some 
respect.  It  is  this  latter  use  that  concerns  choosing 
individuals  who  are  most  likely  to  succeed  at  a particular 
job.  This  use  is  most  appropriate  to  the  earlier  defini- 
tion of  personality  and  the  objective  of  personality  test- 
ing. For  this  purpose,  the  validity  of  a test  can  be 
relatively  low  and  still  be  valuable  (40:131). 


Criticisms  and  Limitations 


Two  Congressional  hearings  were  held  in  1965  to 
review  personality  assessment  practices.  Although  the 
primary  purpose  of  the  hearings  was  to  investigate  the  use 
of  personality  tests  in  federal  government  personnel  selec- 
tion procedures,  many  more  aspects  were  also  investigated. 
The  criticism  of  personality  test  falls  into  two  problem 
areas:  problems  of  usage  and  problems  involving  moral 
issues  (21:203) . 

Usage  problems  involve  test  misuse  and  test  invali- 
dity. Test  misuse  is  the  invalid  use  of  an  assessment  pro- 
cedure that  has  legitimate  uses.  Test  invalidity  involves 
the  use  of  an  assessment  procedure  whose  validity  has  not 
been  established  for  any  purpose  (21:203).  Test  misuse  is 
probably  the  most  serious  problem  in  personality  testing, 
especially  in  personnel  selection  procedures  (36:177). 

A moral  issue  problem  concerns  confidentiality. 

In  some  instances,  individuals  are  concerned  that  the 
information  gained  from  personality  tests  might  be  used 
against  them  in  some  way  (21:204).  Another  moral  issue 
concerns  the  individuals'  right  to  privacy.  Personality 
tests  are  said  to  invade  personal  privacy  (17:16). 

An  important  limitation  of  personality  tests  is 
that  they  tend  to  be  transparent.  The  subject  can  often 
determine  by  reading  the  items  which  response  will  most 
favorably  portray  his  personality  (36:183).  Again,  this 


problem  is  especially  pertinent  to  personnel  selection 
procedures. 

Applications  in  Personnel  Selection 

Psychological  testing  has  been  an  important  facet 
in  personnel  selection  for  industrial  personnel.  Through- 
out the  1950s  much  emphasis  was  placed  on  selection 
criteria  established  by  psychological  tests  (38:282). 
Psychological  tests  when  used  properly  can  reduce  turnover 
and  increase  effectiveness  of  the  selected  work  force 
(26:170).  The  Civil  Rights  Act  of  1964  has  caused  indus- 
try to  place  less  emphasis  on  psychological  tests. 

In  industry  the  most  frequently  used  method  for 
assessing  personality  is  the  self-report  questionnaire. 

The  Minnesota  Multiphasic  Personality  Inventory,  the 
Eysenck  Personality  Inventory,  the  Cattell  16  Personality 
Factor  Questionnaire,  the  Guildford- Zimmerman  Temperament 
Survey,  and  the  Gordon  Personal  Profile  and  Personal 
Inventory  are  the  ones  most  often  used  by  industry  (22:6). 
Other  approaches  to  the  assessment  of  personality,  such 
as  the  Rorschach  ink-blot  test,  are  usually  more  time- 
consuming  and  require  full  psychological  training  for  their 
use  and  interpretation. 

The  Minnesota  Multiphasic  Personality  Inventory 
(MMPI ) has  been  the  most  frequently  used  personality 
inventory  (8:243).  Hospitals  have  used  the  MMPI  with 
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biographical  data  to  select  successful  hospital  aids 
(4:  562)  . 
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Another  personality  measure  frequently  used  by 
industry  is  the  Eysenck  Personality  Inventory.  The  Eysenck 
Personality  Inventory  consists  of  48  items.  This  test  mea- 
sures two  main  traits:  neuroticism  and  extroversion. 
Neuroticism  refers  to  general  emotional  instability. 
Neurotic  breakdown  under  stress  can  be  predicted  with  this 
measure.  Extroversion  refers  to  uninhibited,  outgoing, 
and  sociable  characteristics  a subject  possesses.  Business 
has  used  this  measure  as  a predictor  of  success  of  sales 
personnel  (19:201). 

The  Cattell  16  Personality  Factor  Questionnaire 
and  the  Guildford- Zimmerman  Temperament  Survey  are  similar 
personality  surveys.  These  two  tests  predict  general 
activity,  restraint,  ascendance,  sociability,  emotional 
stability,  objectivity,  friendliness,  thoughtfulness, 
personal  relations,  and  masculinity.  Reliability  and 
validity  of  the  two  tests  are  highly  established  within 
industry  and  business  (19:197-98).  The  Cattell  16  Per- 
sonality Factor  Questionnaire  and  the  Guildf ord-Zimmerman 
Temperament  Survey  have  been  found  to  be  most  valid  and 
useful  to  assess  traits  in  normal  people  for  industry, 
business,  and  school  (19:197-98). 

The  Gordon  Personal  Inventory  and  Personal  Profile 
have  been  found  to  be  useful  in  industrial  selection, 
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appraisal,  vocational  guidance,  personal  counseling,  class- 
room demonstration,  and  basic  research  (15:15;  16:15). 

In  industrial  selection  the  Gordon  Personal  Inventory  and 
Personal  Profile  has  achieved  wide  usage.  Its  main  appeals 
are  its  brevity,  its  relative  resistance  to  distortion, 
its  acceptability  to  the  applicant,  and  most  important,  its 
ability  to  predict  success  in  many  jobs.  The  eight  traits 
measured  by  the  two  Gordon  inventories  are  cautiousness, 
original  thinking,  personal  relations,  vigor,  ascendancy, 
responsibility,  emotional  stability,  and  sociability.  The 
Gordon  Tests  have  repeatedly  demonstrated  their  usefulness 
in  selecting  candidates  for  a particular  job  (15:15;  16:15). 

Industry  and  business  have  used  the  Rorschach  ink- 
blot test  to  a lesser  extent,  because  of  its  questionnable 
reliability  and  validity.  The  Rorschach  test  is  used  to 
assess  level  of  intelligence,  creative  potential,  spontane- 
ity, degree  of  mood  fluctuation,  depression,  euphoria,  and 
the  extent  of  anxiety  (19:276).  Some  success  was  shown 
in  differentiating  the  outstanding  from  ordinary  or  unsatis- 
factory mechanical  workers  on  the  basis  of  their  Rorschach 
responses  (6:21). 

Psychological  testing  has  not  been  prohibited  by 
the  Civil  Rights  Act  of  1964.  The  Act  prohibited  dis- 
crimination in  hiring,  and  only  tests  that  are  discrimina- 
tory in  nature  are  prohibited  (38:283).  To  meet  the 
requirements  of  the  Supreme  Court  and  at  the  same  time 
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satisfy  the  needs  of  industry  and  business,  a simple  rule 
can  be  applied:  people  with  an  equal  probability  of  success 
should  have  an  equal  probability  of  being  hired  (3:174). 

Review  of  POD  Research 

The  DOD  faces  the  same  problems  as  business  and 
industry  in  personnel  selection.  Although  many  studies 
have  been  conducted  concerning  personnel  selection  tech- 
niques used  by  the  DOD,  few  have  been  directed  toward 
personality  assessment.  What  follows  is  a review  of  some 
of  the  most  recent  uses  of  personality  testing  in  the  DOD. 

U.S.  Army  Study.  The  United  States  Army  Personnel 
Research  Office  conducted  a validation  of  a experimental 
electronics  selections  battery.  One  of  the  objectives  of 
their  research  was  to  develop  a measure  for  improving 
assignment  of  personnel  to  electronic  jobs.  A 425-item 
inventory  measure  was  used  by  the  Army  Personnel  Research 
Office  to  measure  such  behavioral  aspects  as  conscientious- 
ness, persistence,  stability,  attitudes  toward  schooling, 
work,  authority,  and  information  regarding  background, 
activities,  and  interests.  Results  of  the  Army’s  valida- 
tion effort  concluded  that  behavioral  aspects  could  be  used 
as  predictors  of  success  in  the  electronics-electrical 
career  field  both  at  high  complexity  and  low  complexity 
levels  (2: 14-18) . 
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Philco  Study.  In  1967  the  Philco  Corporation 


completed  a DOD  study  which  attempted  to  quantify  the  per- 
sonal variables  which  are  necessary  to  good  performance  in 
maintenance  (6:3).  The  project  studied  Air  Force  subjects 
at  the  Cheyenne  Mountain  Complex  near  Colorado  Springs, 
Colorado.  The  objective  of  their  research  effort  was  to 
investigate  the  personnel  element  in  the  maintenance 
environment  and  its  influence  on  downtime.  One  of  the  com- 
ments in  the  report  stated  that  little  work  has  been  done 
in  attempting  to  identify  the  variables  associated  with 
human  performance  in  maintenance  of  military  electronic 
subsystems  (6:13).  Philco,  in  its  research  effort,  used 
three  well  established  personality  inventories  to  quantify 
certain  traits.  Tests  used  by  Philco  were  the  Minnesota 
Multiphasic  Personality  Inventory,  Survey  of  Interpersonal 
Values,  and  the  Strong  Vocational  Interest  Blank.  Through 
the  use  of  these  test  scores,  certain  personality  variables 
were  investigated.  Although  the  conclusions  by  Philco 
Corporation  are  somewhat  vague,  the  report  did  state  that 
certain  of  the  personality  variables  did  show  promise  of 
having  predictive  capability  of  human  performance  in  elec- 
tronic systems  maintenance  (6:8). 

Pilot  Selection.  Application  of  personality  test- 
ing in  other  military  career  fields  has  proven  successful. 
The  United  States  Army  has  found  such  personality  constructs 


as  self-confidence,  activity  level,  and  distractibility  and 
indecisiveness  to  be  good  predictors  of  success  of  pilots 
in  helicopter  training  school  (6:21). 


U.S,  Navy  Study.  A recently  completed  test  conduc- 
ted by  the  Naval  Medical  Research  Institute  studied  certain 
personality  variables  and  their  relationship  to  task  per- 
formance (18:731).  Personality  was  measured  by  the 
Insolence  Scale.  The  Insolence  Scale  was  developed  to 
measure  27  self-description  items.  Affirmative  answers 
to  the  Insolence  Scale  conveyed  the  impression  of  a 
physically  active,  aggressive,  somewhat  hostile  and  reck- 
less personality,  who  early  in  life  became  independent  of 
family  and  grade  school  control,  and  who,  it  is  suspected, 
continues  to  maintain  an  independent  and  rebellious  atti- 
tude toward  most  attempts  at  controlling  his  behavior. 
Subjects  with  high  Insolence  Scale  scores  displayed  lack  of 
interest  in  the  jobs  they  performed  for  this  experiment. 

In  recent  field  application  the  results  parallel  those 
observed  in  the  lab  setting  (18:742). 

An  extensive  literature  review  uncovered  only  a 
few  cases  where  the  DOD  has  used  personality  variables  as 
one  of  the  selection  criteria  for  career  assignments. 
Aptitude  alone  has  remained  the  primary  criterion  for 
assigning  personnel  to  career  fields  (20;  39) . 
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Maintenance  Aid  Research 


Over  the  past  ten  years  DOD  has  placed  much 
emphasis  on  improving  equipment  as  an  approach  to  better 
maintenance.  However,  little  attention  has  been  given  to 
the  area  of  maintenance  information.  No  major  changes  have 
been  made  to  the  conventional  technical  manual  systems. 

Although  some  gradual  improvement  has  occurred,  there  has 
been  no  innovative  effort  to  improve  the  "people"  part  of 
the  maintenance  problem  (7:1;  33:8).  There  have  been  a 
number  of  small-scale  experiments  and  demonstrations  of 
new  ways  of  presenting  information  that  enable  technicians 
to  perform  faster  and  more  accurately  with  less  training. 

However,  the  reactions  to  the  results  of  these  experiments 
have  not  been  overwhelming. 

Although  some  of  these  techniques  appear  to  have 
the  potential  for  greatly  reducing  cost  of  maintenance 
perhaps  by  30%  or  more  and  improving  equipment  avail- 
ability, there  has  been  relatively  little  application 
of  these  research  results  outside  of  the  laboratory 
[33:8] . 

The  research  has  been  primarily  concerned  with  per- 
formance  aids  (33:8).  These  devices  contain  guidance 
information  that  helps  the  technician  perform  a particular 
job  at  hand  and  are  "people"  rather  than  equipment  oriented. 

The  aid  may  be  a very  specific  step-by-step  set  of  direc- 
tions that  require  no  decisions  to  be  made  or  it  may  be 
something  that  gives  a more  general  picture  of  the  system 
and  helps  the  technician  decide  what  to  do  next.  The 
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majority  of  past  research  has  focused  on  various  specific 
or  fully  proceduralized  aids  for  non-troubleshooting  tasks 
(33:9) . 

The  objective  of  past  research  has  been  to  provide 
simple,  complete,  and  current  information  to  the  technician 
without  the  need  for  cross-referencing  and  retention  (33:9). 
Fully  proceduralized  performance  aids  break  up  the  task 
into  easily  understood  steps  and  present  unambiguous  direc- 
tions using  simple  language  and  relevant  illustrations. 

Well  designed  aids  are  based  on  a careful  analysis  of  the 
task  and  take  into  consideration  the  amount  of  training 
and  experience  of  the  personnel  who  will  be  using  them. 

In  contrast,  current  maintenance  documentation  rarely  con- 
siders training  and  experience.  Instead,  it  is  often 
written  before  the  equipment  is  actually  produced  and  is 
not  based  on  the  tasks  required  to  repair  the  equipment 
(33:9).  The  problems  caused  by  this  process  were  revealed 
in  an  analysis  of  the  maintenance  actions  required  for  the 
doppler  radar  system  of  the  C-141.  It  was  found  that  the 
isolation  and  repair  of  one  malfunction  required  reference 
to  165  pages  of  8 different  documents.  If  no  mistakes 
were  made,  41  changes  in  document  location  were  required 
(33:10) . 

Fully  proceduralized  aids  have  been  developed  for 
troubleshooting  tasks,  but  most  of  the  research  on  trouble- 
shooting has  involved  decision  aids  (24:29).  These  aids 


help  the  technician  understand  the  system  and  formulate  a 
strategy  for  isolating  the  fault.  Most  of  the  decision 
aids  have  included  variations  of  the  Maintenance  Depen- 
dency Chart  (MDC) , which  can  vary  greatly  in  complexity 
and  which  indicates  what  elements  are  interrelated  (24:29). 
Past  research  projects  concerning  maintenance  aids  have 
been  summarized  a number  of  times  (6:11-30;  10:3-17; 
32:8-19).  What  follows  is  a review  of  those  projects  most 
closely  related  to  this  research  effort. 

FORECAST 

Project  FORECAST  has  been  concerned  primarily  with 
the  training  demands  posed  by  new  weapons  systems.  The 
objective  of  the  research  was  to  develop  and  test  methods 
for  analyzing  an  electronic  weapon  system  to  define  a set 
of  skills  and  the  knowledge  necessary  for  operating  and 
maintaining  the  system  (34:43).  The  aids  produced  are 
decision  type  aids  such  as  troubleshooting  block  diagrams, 
waveform  guides,  and  blocked  schematics.  Such  aids  are 
used  during  training  to  convey  system  understanding  and  to 
teach  troubleshooting  (33:15). 

In  1958,  an  experimental  12-week  FORECAST  training 
program  was  conducted  for  Army  M-33  antiaircraft  fire  con- 
trol system  electronic  technicians  (34:43).  A comparison 
of  performance  was  made  between  20  experimental  subjects 
for  the  12-week  program  and  17  subjects  from  the  normal 
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30-week  course.  The  experiment  used  89  troubleshooting 
tasks  administered  over  a 9-day  period.  The  mean  test 
scores  and  standard  deviations  show  very  little  difference 
in  the  performance  of  the  two  groups  (34:43-46).  The 
important  difference  was  the  length  of  training.  The 
group  which  received  the  shortened  12-week  course  per- 
formed as  well  as  the  group  which  received  the  standard 
30- week  course. 

Another  test  was  conducted  in  1963  with  98  sub- 
jects using  Navy  Loran  equipment.  Twelve  subjects  had 
conventional  training  and  manuals.  The  remaining  number 
had  FORECAST  training  and  manuals.  The  FORECAST  subjects 
identified  an  average  of  three  times  as  many  malfunctions 
as  the  conventional  subjects  (33:16). 

JOBTRAIN 

JOBTRAIN  was  the  result  of  research  by  the  Human 
Resources  Research  Organization  (HumRRO) . The  study  was 
designed  to  develop  and  evaluate  methods  for  producing  a 
combination  of  training  and  job  aids  for  maintenance  of 
electronic  communication  equipment  such  that  less  training 
time  would  be  required  than  under  the  standard  course  and 
manuals  (13:3).  The  aid  and  training  course  was  compared 
with  standard  documents  and  training  in  1962  at  Fort 
Gordon,  Georgia.  The  subjects  were  radio  relay  and  car- 
rier repair  technicians.  The  control  group  consisted  of 


31 


19  trainees  who  had  attended  the  normal  25-week  school  and 
used  conventional  documents.  The  experimental  group  con- 
sisted of  21  trainees  who  had  received  the  11-week  JOBTRAIN 
course  and  used  the  JOBTRAIN  aid.  The  two  groups  were 
given  18  problems  to  troubleshoot  (13:20-29).  The  test 
results  indicated  that  the  two  groups  performed  equally 
well  since  no  statistically  significant  differences  were 
found.  It  was  concluded  that  the  JOBTRAIN  training  and 
aids  were  as  effective  as  traditional  training  and  manuals 
and  that  a 50  percent  reduction  in  training  time  could 
result  from  the  JOBTRAIN  approach  (33:17). 

MAINTRAIN 

MAINTRAIN,  another  HumRRO  project,  was  an  effort 
to  develop  an  improved  manual  for  use  in  troubleshooting 
complex  electronic  equipment.  The  manual  was  tested  by 
having  recent  missile  school  graduates  troubleshoot  44 
faults  with  Nike  Ajax  Missiles.  The  subjects  were  matched 
on  the  basis  of  Air  Defense  School  grades  and  divided  into 
two  groups  of  eight.  One  group  used  conventional  manuals 
and  the  other  used  the  MAINTRAIN  manual.  The  group  using 
the  MAINTRAIN  manual  located  42  percent  more  malfunctions 
in  41  percent  less  time  (33:17). 

SIMS 

Symbolic  Integrated  Maintenance  System  (SIMS)  is 
a program  which  also  deals  with  maintenance  manuals.  A 
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military  specification  was  written  to  set  forth  require- 
ments for  this  type  of  technical  manual  for  the  maintenance 
and  repair  of  electronic  systems  and  training  of  military 
personnel  (35:79).  In  research  conducted  in  1964,  SIMS 
manuals  were  constructed  for  a Coast  Guard  radar  set.  An 
experiment  using  42  subjects  divided  into  two  groups  showed 
that  the  group  using  the  SIMS  manuals  identified  the  three 
test  malfunctions  96  percent  of  the  time.  The  group  using 
conventional  manuals  had  only  a 70  percent  success  rate. 
Additionally,  the  SIMS  group  took  only  half  as  much  time 
to  troubleshoot  (35:80-85). 

PIMP 

Project  PIMO  (Presentation  of  Information  for 
Maintenance  and  Operations)  has  been  the  most  extensive 
effort  in  the  job  aid  area  (27:9).  PIMO  consists  of  fully 
proceduralized  aids  for  non-troubleshooting  maintenance 
tasks  and  simplified  maintenance  dependency  charts  for 
troubleshooting.  The  aids  used  a fixed  format  and  a pre- 
ferred verb  list  with  a limited  number  of  steps  per  page. 
Between  August  1968  and  April  1969,  a test  was  conducted  at 
Charleston  AFB,  South  Carolina,  to  determine  the  effective- 
ness of  PIMO.  Experienced  and  inexperienced  technicians 
performed  maintenance  on  C-141A  aircraft  using  PIMO  job 
guidance  presented  in  audio  visual  and  booklet  modes  (27: 
9-10).  Performance  was  measured  in  terms  of  time  to 


perform  tasks  and  procedural  errors.  The  performance  was 
compared  with  the  performance  on  the  same  jobs  by  experi- 
enced technicians  performing  in  the  normal  manner.  Several 
conclusions  were  drawn  from  the  test  results.  First, 
both  experienced  and  inexperienced  technicians  showed 
strong  evidence  of  learning  while  performing  with  the  PIMO 
Job  Guides.  Generally  their  performance  times  reduced 
(after  7 or  8 trials)  to  approximately  the  same  level  as 
the  very  experienced  technician  performing  under  "normal" 
conditions.  During  the  learning  period,  all  performance 
was  error-free  (27:13).  This  is  not  the  case  when  tech- 
nicians perform  in  their  regular  fashion.  Secondly, 
inexperienced  technicians  performed  as  well  as  experienced 
technicians  when  both  used  PIMO  non-troubleshooting  aids. 
During  the  experiment,  the  users  expressed  an  overwhelm- 
ingly positive  reaction  to  PIMO. 

Only  experienced  personnel  were  used  in  testing 
the  PIMO  troubleshooting  aids.  There  was  an  11  percent 
reduction  in  time  using  PIMO  as  opposed  to  standard  pro- 
cedures. Forty  troubleshooting  actions  were  performed 
with  PIMO  with  one  error  while  forty-one  actions  were  per- 
formed using  standard  procedures  with  12  errors  (27:19). 

Naval  Air  Systems  Command  Project 

In  1970  a field  test  of  fully  proceduralized  non- 
troubleshooting aids  was  conducted  for  the  Naval  Air 
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Systems  Command  onboard  the  USS  Saratoga  and  at  the  Naval 
Air  Station,  Oceana.  The  test  involved  19  organizational 
maintenance  tasks  on  the  F-4J  (28:6).  Fifty-two  tech- 
nicians performed  a total  of  635  maintenance  actions. 
Portions  of  the  existing  Maintenance  Information  Manuals 
that  were  relevant  to  the  malfunctions  used  in  the  test 
were  collected,  checked  for  accuracy,  and  packaged  for 
each  maintenance  task  so  that  the  content  of  the  tradi- 
tional material  would  be  as  available  as  the  job  guides. 
During  the  test,  the  experienced  personnel  had  the  option 
of  using  or  not  using  the  exerpts  from  the  manuals.  The 
inexperienced  personnel  were  required  to  use  them  (33:26). 
Using  quality  assurance  standards  as  a guide,  experienced 
and  inexperienced  personnel  using  standard  procedures  per- 
formed satisfactorily  74  and  63  percent  of  the  time, 
respectively.  When  using  the  job  aids,  experienced  and 
inexperienced  personnel  performed  satisfactorily  100  and 
94  percent  of  the  time,  respectively.  These  results 
showed  that  inexperienced  personnel  performing  with  job 
aids  were  20  percent  better  than  experienced  personnel 
using  standard  procedures  (33:26). 


AFHRL  Project 


The  Air  Force  Human  Resources  Laboratory  (AFHRL) 


Ohio,  recently  completed  a field 
The  test  involved  three  different 
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aids:  Logic  Tree  Troubleshooting  Aids  (LTTA) , Fully  Pro- 
ceduralized  Troubleshooting  Aids  (FPTA) , and  the  standard 
Air  Force  Technical  Order  (T.O.),  The  most  significant 
difference  between  the  types  of  aids  is  the  degree  of  pro- 
ceduralization  (29:4).  The  FPTAs  contain  the  highest 
degree  of  proceduralization  and  T.O.s  the  lowest,  with  the 
LTTAs  somewhere  in  between  the  two. 

The  standard  Air  Force  T.O.  contains  much  of  the 
information  needed  to  successfully  troubleshoot  the  equip- 
ment. However,  the  technician  must  decide  what  data  he 
will  use  and  how  and  when  it  will  be  used.  The  manuals 
are  prepared  assuming  that  the  technician  has  experience 
and  training  in  the  use  of  test  equipment,  in  locating 
most  parts  within  the  equipment,  and  interpreting  sche- 
matic diagrams  (29:5). 

The  LTTAs  are  more  proceduralized  than  conventional 
T.O.s.  Specific  instructions  are  provided  both  in  the 
checkout  procedure  and  in  the  troubleshooting  procedure. 

The  troubleshooting  logic  trees  give  exact  procedures  to 
be  performed  and  ask  questions  about  results  obtained 
while  monitoring  voltages,  waveforms,  and/or  resistances 
at  specific  points  in  the  circuit  (29:6).  The  yes  or  no 
answer  to  these  questions  provides  the  logic  in  the  fault 
isolation  procedure.  Each  logic  path  ends  with  instruc- 
tions to  replace  a faulty  component.  Sometimes  a path 
will  first  identify  the  faulty  stage  or  assembly  and  will 
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direct  the  technician  to  perforin  the  checkout  procedure 
for  that  stage  or  assembly. 

LTTAs  are  similar  to  standard  T.O.s  in  that  they 
are  prepared  for  the  most  experienced  technician.  The 
technicians  are  assumed  to  be  able  to  properly  choose  and 
use  test  equipment,  locate  most  parts  and  test  points,  and 
read  and  interpret  functional  block  diagrams  (29:6). 

FPTAs  provide  complete  step-by-step  instructions 
for  both  checkout  and  troubleshooting  tasks.  They  are 
designed  for  use  by  both  experienced  and  inexperienced 
technicians.  All  specific  test  equipment  and  tools  are 
called  out  in  each  section  as  they  are  required.  Each 
prime  hardware  item  and  the  more  complex  test  equipment 
mentioned  in  the  aid  are  accompanied  by  a call-out  number 
which  is  keyed  to  an  illustration  of  the  items  appearing 
on  that  same  page  or  a facing  page  (29:8).  For  test  equip- 
ment, the  illustrations  appear  in  a fold-out  page  at  the 
back  of  each  section.  The  FPTAs  are  complemented  by  main- 
tenance support  information  manuals.  These  manuals  con- 
tain instructions  on  test  equipment  operation,  special 
testing  procedures,  and  wiring  diagrams.  They  are  not 
required  for  use  with  the  FPTAs  but  are  available  if  needed. 
To  facilitate  use  of  the  aids  by  experienced  personnel, 
the  basic  checks,  expected  readings,  and  tolerances  are 
underlined.  Underlined  information  permits  the  experienced 


technician  to  rapidly  scan  the  test  and  focus  his  atten- 
tion on  important  data  only  (29:8). 


A carefully  controlled  experiment  was  conducted 
under  field  conditions  to  assess  the  merits  of  T.O.s, 

LTTAs , and  FPTAs  in  locating  and  correcting  problems  in 
two  electronic  subsystems  of  the  C-141A  aircraft.  For 
the  purposes  of  the  test,  three  experience  levels  of  Air 
Force  technicians  were  sampled;  recent  technical  school 
graduates,  field-experienced  technicians  with  six  months 
or  less  experience,  and  technicians  with  more  than  six 
months  experience.  Since  the  technical  school  graduates 
were  not  normally  expected  to  be  able  to  perform  with 
T.O.s,  they  were  not  measured  with  this  aid.  For  this 
reason  the  test  was  conducted  with  Keesler  technical 
school  graduates  as  a separate  analysis  (29:10). 

The  results  of  the  AFHRL  study  are  as  follows 

(29:45)  : 

1.  The  quality  of  performance  was  net  affected  by 
the  use  of  FPTA  or  LTTA  for  different  experience  levels. 

Low  experience  level  subjects  who  used  the  T.O.  had  lower 
performance  ratings. 

2.  Total  number  of  problems  solved  were  greater 
using  the  FPTA  and  LTTA  as  compared  with  number  of  problems 
solved  using  the  standard  T.O. 

3.  Problems  were  solved  with  more  consistency 
with  either  the  LTTA  or  FPTA. 
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4.  Spare  parts  consumption  was  significantly  lower 
with  the  LTTA  or  FPTA. 

5.  The  T.O.  was  viewed  as  a much  more  difficult 
aid  to  use  as  compared  to  the  FPTA  and  LTTA. 

6.  At  all  levels  of  complexity  of  troubleshooting 
examined,  the  T.O.  was  rejected  in  favor  of  either  the 
FPTA  or  LTTA.  The  FPTA  was  the  most  favored  by  the  sub- 
jects. 


Implications  for  Research 
The  little  research  that  has  been  conducted  con- 
cerning the  maintenance  "people"  problem  has  dealt  with 
maintenance  aids  in  an  attempt  to  improve  the  information 
transfer  to  the  technician  on  the  job.  Improved  mainte- 
nance aids  have  been  shown  to  have  potential  for  cost 
reductions  in  personnel  areas. 

The  maintenance  aid  research,  while  very  important, 
does  not  approach  the  problem  of  personnel  selection  and 
classification.  An  extensive  literature  review  has 
revealed  very  little  work  done  in  an  attempt  to  improve 
personnel  selection  and  classification  procedures  within 
the  DOD.  The  literature  review  did  show  that  personality 
testing  is  one  method  which  has  been  commonly  used  for 
personnel  selection  in  business  and  industry. 

Personality  testing,  although  somewhat  limited  by 
the  Civil  Rights  Act  of  1964,  can  be  an  effective  tool  in 
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predicting  job  success  (12:217).  The  implication  is  that 
if  certain  personality  traits  could  be  identified  as  pre- 
dictors of  success  in  maintenance  jobs,  the  quality  of  the 
maintenance  work  force  could  be  improved  through  improved 
selection  procedures.  The  combination  of  improved  selec- 
tion procedures  and  improved  maintenance  aids  would  have  a 
great  potential  for  savings  in  maintenance  costs. 

Research  Propositions  and  Hypotheses 
The  following  is  a listing  of  the  propositions  and 
associated  hypotheses  which  are  derived  from  the  literature 
review  and  the  research  objectives. 

Proposition  1:  Individuals  who  perform  well  using 

one  type  of  troubleshooting  aid,  tend  to  perform  well  using 
the  other  types  of  troubleshooting  aids. 

Hypothesis  No.  1:  For  technicians  with  no 

experience,  using  the  FPTA  and  the  LTTA,  there  are  differ- 
ences among  the  mean  ranks  of  the  subjects'  performance. 

Hypothesis  No.  2:  For  technicians  with  six 

months  or  less  experience,  using  the  FPTA,  LTTA,  and  the 
T.O.,  there  are  differences  among  the  mean  ranks  of  the 
subjects'  performance. 

Hypothesis  No.  3:  For  technicians  with  more 

than  six  months  experience,  using  the  FPTA,  LTTA,  and  the 
T.O.,  there  are  differences  among  the  mean  ranks  of  the 
sub j ec t s ' performance . 

40 


A 


Proposition  2:  A significant  difference  exists 

between  the  troubleshooting  performance  of  technicians  of 
different  experience  levels. 

Hypothesis  No.  4:  There  is  a difference  in  the 

troubleshooting  performance  of  technicians  with  no  experi- 
ence, six  months  or  less  experience,  and  more  than  six 
months  experience. 

Proposition  3:  A relationship  exists  between  spe- 

cific personality  traits  and  demonstrated  troubleshooting 
ability. 

Hypothesis  No.  5:  A relationship  exists 

between  Ascendancy  and  demonstrated  troubleshooting  ability. 

Hypothesis  No.  6:  A relationship  exists 

between  Responsibility  and  demonstrated  troubleshooting 
ability. 

Hypothesis  No.  7:  A relationship  exists 

between  Emotional  Stability  and  demonstrated  troubleshoot- 
ing ability. 

Hypothesis  No.  8:  A relationship  exists 

between  Sociability  and  demonstrated  troubleshooting 
ability. 

Hypothesis  No.  9:  A relationship  exists 

between  Cautiousness  and  demonstrated  troubleshooting 
ability. 


Hypothesis  No.  10:  A relationship  exists 

between  Original  Thinking  and  demonstrated  troubleshooting 
ability. 

Hypothesis  No.  11.  A relationship  exists 
between  Personal  Relations  and  demonstrated  troubleshooting 
ability. 

Hypothesis  No.  12:  A relationship  exists 

between  Vigor  and  demonstrated  troubleshooting  ability. 

Proposition  4:  There  is  a positive  relationship 

between  AQE  aptitude  measurement  scores  and  demonstrated 
troubleshooting  ability. 

Hypothesis  No.  13:  There  is  a positive  rela- 

tionship between  Mechanical  aptitude,  as  measured  by  the 
AQE,  and  demonstrated  troubleshooting  ability. 

Hypothesis  No.  14:  There  is  a positive  rela- 

tionship between  Administrative  aptitude,  as  measured  by 
the  AQE,  and  demonstrated  troubleshooting  ability. 

Hypothesis  No.  15:  There  is  a positive  rela- 

tionship between  Electronics  aptitude,  as  measured  by  the 
AQE,  and  demonstrated  troubleshooting  ability. 

Hypothesis  No.  16:  There  is  a positive  rela- 

tionship between  General  aptitude,  as  measured  by  the  AQE, 
and  demonstrated  troubleshooting  ability. 
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CHAPTER  III 


METHODOLOGY 

This  research  used  the  data  which  was  collected 
during  the  previously  described  AFHRL  research  project 
(29;  30) . The  first  section  of  this  chapter  describes  the 
population  and  sample  for  which  data  was  collected.  The 
remainder  of  the  chapter  describes  the  data  collection  pro- 
cedures and  the  statistical  analysis  techniques  for  each  of 
the  research  propositions. 

Description  of  the  Population  and  Sample 
The  population  for  the  AFHRL  project  consisted  of 
Air  Force  enlisted  personnel  in  the  328X4  Air  Force  Spe- 
ciality Code  (AFSC)  (Avionics  Inerrtal  and  Radar  Naviga- 
tional Systems  Specialists) . A sample  of  54  personnel  in 
this  AFSC  was  studied  (30:16).  This  total  of  54  was 
divided  into  three  subject  groups  based  on  experience 
level.  The  first  group  consisted  of  18  3-level  airmen  who 
were  recent  graduates  of  the  Keesler  Technical  Training 
Center  (KTTC)  Course  3ABR32834.  These  subjects  were  first- 
term  enlisted  airmen  who  had  completed  basic  training  and 
who  went  directly  into  technical  training  at  KTTC.  They 
had  no  field  experience.  The  airmen  were  retained  at 
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Keesler  for  a two-week  period  beyond  course  graduation  and 
entered  as  subjects  for  the  experiment  (30:14). 

Graduation  dates  and  sizes  of  the  classes  used 
were:  7 October,  7 graduates;  21  October,  9 graduates; 

4 November,  9 graduates.  Each  graduate  was  assigned  to  an 
initial  evaluation  group  by  means  of  a random  selection 
process  (30:19).  All  graduating  classes  were  used  in  total, 
providing  25  subjects  on  which  complete  data  was  collected. 
Since  the  experiment  required  a sample  size  of  only  18  sub- 
jects, the  decision  was  made  to  drop  the  7 October  class 
from  the  analysis  (30:29).  The  decision  to  drop  the  first 
class  was  based  on  the  fact  that  no  "dry-runs"  had  been 
conducted  with  the  test  equipment  prior  to  evaluating  the 
first  subjects;  therefore,  any  unusual  equipment  problems 
or  experimental  bias  would  have  been  more  likely  during  the 
evaluation  of  the  first  group  (31).  The  two  classes  used 
in  the  analysis  were  believed  to  be  a representative  sam- 
ple. Since  all  assignments  to  the  KTTC  course  were  based 
on  the  same  selection  procedures  and  criteria,  there  was 
no  reason  to  believe  that  any  particular  class  differed 
significantly  from  any  other  class. 

The  two  remaining  subject  groups  were  obtained 
from  Air  Force  enlisted  personnel  in  the  328X4  AFSC 
assigned  to  operational  units  of  the  Military  Airlift 
Command  (MAC)  and  performing  maintenance  on  the  AN/ASN-35 
(computer)  and  AN/APN-147  (radar)  systems  in  C-141A 
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aircraft  (30:15).  Each  group  consisted  of  18  subjects  and 
were  classified  according  to  length  of  experience.  One 
group  of  18  were  airmen  who  had  either  been  awarded  the 
5- level  of  the  AFSC  within  the  last  six  months  or  for  whom 
award  was  imminent  (30:16).  This  group  had  six  months  or 
less  experience  on  the  system.  The  remaining  18  subjects 
were  5- level  airmen  with  more  than  six  months  experience. 
Three  MAC  bases  were  visited  to  obtain  the  required  number 
of  subjects  for  these  two  groups. 

The  choice  of  bases  to  be  visited  to  collect  data 
was  limited  by  two  considerations  (31) . First,  the  experi- 
ment was  confined  to  MAC  bases  within  the  continental 
United  States.  Secondly,  certain  U.S.  bases  (Charleston 
AFB,  Norton  AFB,  and  Altus  AFB)  were  not  considered  since 
studies  concerning  maintenance  aids  had  already  been  con- 
ducted at  these  locations  and  the  results  of  any  additional 
experiments  might  have  been  contaminated.  Within  these 
limitations,  an  attempt  was  made  to  obtain  the  widest 
possible  range  of  geographical  locations.  The  bases 
chosen  were:  McChord  AFB,  Washington;  Travis  AFB,  Cali- 

fornia; and  McGuire  AFB,  New  Jersey  (20:17). 

Each  of  the  three  bases  provided  a list  of  per- 
sonnel with  the  appropriate  AFSC  and  experience  level.  The 
subjects  for  the  experiment  were  randomly  selected  from 
these  lists  (31).  Due  to  the  nature  of  Air  Force  assign- 
ment policies  and  procedures,  it  was  believed  that  all 
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elements  of  the  population  of  interest  were  as  likely  to  be 
assigned  to  any  of  the  three  bases  visited  as  any  other 
base;  therefore,  the  sample  was  believed  to  be  representa- 
tive. Since  the  actual  experiment  resulted  in  a wide  range 
of  performance,  there  was  no  reason  to  believe  that  the 
sample  was  not  representative  (31) . 

A sample  size  of  18  subjects  was  decided  upon  for 
two  basic  reasons  (31).  First,  it  was  decided  to  design 
the  experiment  around  a small  sample  because  of  the  cost 
and  time  involved  in  collecting  the  data.  Secondly,  the 
specific  number  of  18  was  chosen  to  balance  the  experi- 
mental design.  Since  there  were  two  types  of  aids  to  be 
evaluated  in  one  group  and  three  types  in  the  other  two 
groups,  a sample  of  18  allowed  an  equal  number  of  subjects 
to  be  evaluated  with  each  type  of  aid  in  all  three  groups. 

Research  Proposition  No.  1 
Testing  Environment 

An  Air  Force  trailer  configured  to  include  AN/ASN- 
35  and  AN/APN-147  bench  mock-ups  and  a mock-up  of  the 
C-141A  cockpit  was  taken  to  four  separate  Air  Force  bases: 
Keesler  AFB,  Mississippi;  McChord  AFB,  Washington;  Travis 
AFB,  California;  and  McGuire  AFB,  New  Jersey  (20:17).  The 
trailer  served  as  the  experimental  setting  for  the  con- 
trolled evaluation. 
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Since  it  was  difficult  to  obtain  access  to  C-141 
aircraft  for  use  with  the  tests,  the  mock-up  of  the  C-141 
cockpit  and  avionics  equipment  bays  was  developed  and 
installed  in  the  test  trailer  (30:  17).  The  cockpit  mock-up 
provided  a means  of  simulating  the  troubleshooting  tasks 
without  requiring  access  to  an  aircraft.  It  consisted  of 
mock-ups  of  all  instrument  panels  in  the  cockpit.  All 
AN/APN-147  and  AN/ASN-35  components  were  "live"  and  func- 
tioned in  the  same  manner  as  the  actual  cockpit  (30:17). 

All  other  components  on  the  instrument  panels  were  repre- 
sented by  photographs. 

Technical  Accuracy  of  Trouble- 
shooting Test  Problems 

The  equipment  problems  against  which  each  of  the 
three  types  of  troubleshooting  aids  were  evaluated  were 
selected  after  an  extensive  review  of  worldwide  maintenance 
actions  on  the  AN/APN-147  and  AN/ASN-35  systems  for  calen- 
dar year  1974.  The  source  of  the  maintenance  action  data 
was  Air  Force  Logistics  Command  reports  RCS:  LOG-MMO  (AR) 

7167  and  LOG-MMO  (AR)  7168  (30:13).  The  data  contained  in 
these  reports  were  summarized  to  provide  identification  of 
maintenance  areas  which  appeared  to  be  logical  and  repre- 
sentative selections  for  development  of  troubleshooting 
test  problems.  In  order  to  provide  field  verification  of 
the  validity  of  the  data  summarizations,  the  results  were 
discussed  in  detail  with  maintenance  personnel  at 
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Charleston  AFB,  South  Carolina  (30:13).  These  discussions 
revealed  detailed  information,  not  available  in  the  AFLC 
reports,  concerning  specific  parts  or  components  which  were 
contributing  most  of  the  maintenance  problems  of  the  systems 
under  study.  This  information  was  considered  in  selecting 
the  troubleshooting  test  problems.  A total  of  30  malfunc- 
tions were  selected  and  located  on  an  appropriate  system 
schematic  to  verify  that  the  problems  selected  were  satis- 
factorily distributed  throughout  the  system.  Fifteen  of 
these  malfunctions  were  then  selected  for  incorporation 
into  the  experimental  design. 

Testing  Procedures 

Keesler  Subjects.  These  airmen  were  available  on 
a full-time  basis  for  two  weeks  following  graduation  from 
Course  3ABR32834.  Each  subject  was  assigned  to  an  initial 
evaluation  group  by  means  of  a random  selection  process 
(30:19).  The  subjects  were  given  remedial  training  courses 
to  familiarize  them  with  the  13  items  of  test  equipment  to 
be  used  for  performing  troubleshooting  procedures  in  the 
evaluation. 

A total  of  15  equipment  faults  were  selected  as 
test  problems  in  the  troubleshooting  evaluation.  However, 
since  a decision  had  been  made  not  to  test  the  Keesler 
subjects  using  technical  orders,  one  of  the  15  problems 
was  eliminated.  Thus,  the  number  of  problems  was  balanced 


48 


so  that  an  equal  number  could  be  evaluated  with  each  of  the 
two  troubleshooting  aids:  seven  with  the  LTTA  and  seven 

with  the  FPTA  (30:20).  Each  equipment  fault  was  isolated 
an  equal  number  of  times  with  both  LTTA  and  FPTA  documenta- 
tion. The  type  of  troubleshooting  aid  used  first  by  the 
subjects  was  determined  by  random  assignment. 

For  purposes  of  the  AFHRL  study,  the  14  problems 
were  further  divided  into  maintenance  levels  for  trouble- 
shooting tasks.  Ten  of  the  problems  were  representative 
of  intermediate  level  maintenance  troubleshooting  tasks 
and  the  remaining  four  were  representative  of  organiza- 
tional level  maintenance  troubleshooting  tasks  (30:20). 

The  organizational  maintenance  level  refers  to  those 
problems  which  might  be  encountered  in  flightline  mainte- 
nance and  the  intermediate  level  refers  to  problems 
encountered  in  shop  maintenance.  Since  this  analysis  is 
not  concerned  with  the  maintenance  level  of  troubleshooting 
tasks,  both  types  of  problems  were  considered  equally  in 

terms  of  performance.  3 

For  each  troubleshooting  problem,  the  test  adminis- 
trator prepared  the  test  equipment  by  completing  the  "pre- 
test setup"  according  to  the  Test  Administrator's  Guide 

] 

which  was  provided  for  him  (30:40).  He  also  verified  that 
the  necessary  support  materials  for  each  problem  were  avail- 
able. The  subject  was  then  brought  into  the  troubleshooting 
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situation  and  given  an  instructions  sheet  and  the  work 
order  which  described  the  problem  he  was  to  troubleshoot. 

The  troubleshooting  evaluation  was  conducted  with 
one  test  administrator  to  one  subject.  The  test  adminis- 
trator recorded  data  on  subject  performance  using  such 
measures  as  time,  errors,  parts  consumed,  successful  com- 
pletion of  each  section,  correctness  in  following  the  tech- 
nical documentation,  and  success  in  troubleshooting  the 
problem  (30:40).  The  test  administrator  was  instructed 
not  to  answer  procedural  questions  during  the  test. 

In  an  operational  situation,  troubleshooting  fre- 
quently involves  removal  and  replacement  of  suspected  parts 
and  components,  which  equates  to  a trial  and  error  method 
of  isolating  the  fault  (30:41).  Since  such  a procedure 
would  have  exposed  the  test  equipment  to  unnecessary  abuse 
and  possibly  caused  total  failure,  at  no  time  was  the 
subject  allowed  to  repair  any  of  the  equipment.  In  the 
test  situation,  when  a subject  indicated  that  he  would 
remove  and  replace  a specific  component  which  was  not  at 
fault,  he  was  instructed  to  assume  that  he  had  done  so  and 
to  continue.  Standard  times  were  derived  for  removal  and 
replacement  of  the  so  identified  components  and  these  times 
were  added  to  subject  times  to  obtain  a total  elapsed  time 
to  fault  isolation  (30:41). 

MAC  Subjects:  The  subjects  for  the  second  two 

groups  were  obtained  from  operational  units  at  McChord, 
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Travis,  and  McGuire  Air  Force  Bases  (30:42).  Since  the 
units  at  these  three  sites  were  operating  on  a three- shift 
day,  subjects  from  each  of  the  three  shifts  were  used  to 
obtain  the  required  sample  sizes.  Each  subject  partici- 
pated in  the  evaluation  for  four  hours  each  day  for  a two- 
week  period. 

Proficiency  tests  evaluating  a subject’s  ability 


to  use  specific  items  of  the  test  equipment  were  adminis- 
tered on  a selective  basis.  Since  it  was  believed  that  an 


experienced  technician  could  realistically  assess  his  own 
familiarity  with  a specific  item  of  test  equipment,  each 
subject  was  presented  with  a list  of  test  equipment  and 
asked  to  rank  his  ability  to  use  each  of  the  items  (30:43). 
Proficiency  tests  were  administer ad  only  when  the  subject 
indicated  a questionable  proficiency  or  when  past  experi- 
ence indicated  that  increased  familiarity  with  an  item  of 
test  equipment  was  needed. 

The  MAC  subjects  were  exposed  to  the  full  set  of 
15  equipment  faults.  Again,  the  faults  were  divided  into 
maintenance  levels  with  four  faults  being  representative 
of  organizational  level  maintenance  troubleshooting  tasks 
and  the  remaining  11  being  representative  of  intermediate 
level  maintenance  troubleshooting  tasks  (30:44).  As  with 
the  Keesler  group,  the  maintenance  level  was  not  of  primary 
concern  in  this  analysis.  Each  fault  was  isolated  an 
equal  number  of  times  with  the  T.O.,  LTTA,  and  FPTA 
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documentation.  The  order  in  which  the  documentation  types 
(T.O. , LTTA,  or  FPTA)  were  assigned  to  problems  was  deter- 
mined by  random  selection. 


The  testing  procedure  followed  at  each  MAC  base 
was  dependent  upon  the  agreements  reached  with  base  per- 
sonnel. Once  the  base  requirements  were  established,  a 
procedure  was  designed  so  that  a standard  approach  to  each 
subject  on  the  base  was  maintained  (30:57).  To  the  extent 
possible,  the  procedure  at  each  MAC  base  was  the  same  and 
conformed  as  nearly  as  was  practical  to  the  Keesler  testing 
procedure.  Variances  in  procedures  from  base  to  base  were 
primarily  administrative  with  respect  to  access  to  tech- 
nicians and  physical  facilities  (30:57).  In  all  cases, 
once  the  subject  was  placed  in  the  troubleshooting  situa- 
tion, he  was  treated  in  an  identical  manner  to  Keesler 
subjects  and  the  same  measures  of  performance  were  obtained. 

Description  of  Variables 

The  variables  of  primary  concern  in  Research 
Proposition  No.  1 are  troubleshooting  performance  and  type 
of  troubleshooting  aids.  An  intensive  review  of  related 
literature  failed  to  reveal  any  standard  method  of  quanti- 
fying troubleshooting  performance.  From  the  review  of 
research  dealing  with  troubleshooting  tasks  and  interviews 
with  experts,  it  seems  that  the  most  generally  accepted 
method  of  measuring  troubleshooting  performance  is  in  terms 
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of  time  (6:13;  20;  21;  22:38;  39).  Therefore,  for  pur- 
poses of  this  research,  troubleshooting  performance  is 
defined  as  an  average  time  per  troubleshooting  problem 
solved.  The  measure  was  derived  from  the  performance  data 
collected  for  the  AFHRL  project.  Total  time  was  found  by 
summing  the  times  spent  on  each  problem  (whether  solved  or 
not)  with  a particular  type  of  troubleshooting  aid.  The 
total  time  included  the  additional  time  added  to  a sub- 
ject's time  for  incorrect/unnecessary  replacement  of  parts 
or  components.  The  total  time  was  then  divided  by  the 
number  of  problems  solved  correctly  with  that  type  of  aid 
to  derive  an  average  time  per  problem  solved  with  that  aid. 
This  performance  measure  was  computed  for  each  subject  in 
the  three  groups  with  each  type  of  aid  used.  The  formula- 
tion of  the  performance  index  was  as  follows: 


where  PI  = performance  index, 

T = times  spent  on  all  problems  and  average  times 
to  remove  and  replace  components  incorrectly 
identified  as  faulty,  and 

r.Q  = number  of  problems  solved  correctly  (nQ  >_  1)  . 

The  types  of  troubleshooting  aids  used  were  LTTAs , 
FPTAs , and  T.O.s,  which  are  described  in  the  summary  of 
the  AFHRL  project  (Chapter  II,  pp.  35-38). 
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Design  to  Test  Research 
Proposition  No.  1 


The  first  research  proposition  (i.e.,  individuals 
who  perform  well  using  one  type  of  troubleshooting  aid, 
tend  to  perform  well  using  the  other  types  of  troubleshoot- 
ing aids),  was  tested  by  means  of  three  separate  hypothesis 
tests;  one  hypothesis  test  for  each  of  the  three  experience 
levels  sampled.  The  nonparametric  Friedman  two-way  analysis 
of  variance  by  ranks  test  was  used  to  test  each  hypothesis. 
The  null  hypothesis  {H  Q)  for  each  test  was  that  there  is 
no  difference  in  the  mean  ranks  of  subjects  using  various 
troubleshooting  aids.  The  alternate  hypothesis  (H ^ ) was 
that  there  are  differences  among  the  mean  ranks  of  perform- 
ances of  individuals.  The  zero-experience-level  group 
(Keesler  graduates)  were  ranked  according  to  performance 
with  only  the  LTTA  and  the  FPTA.  The  six-months-or-less- 
experience  group  and  the  more-than-six-months-experience 
group  were  ranked  using  all  three  aids. 

Friedman  Two-Way  Analysis  of  Variance  by  Ranks 
Test.  The  Friedman  Test  required  that  the  sample  data  be 
in  at  least  an  ordinal  scale  (37:166).  The  sample  data 
for  this  research  consisted  of  the  previously  described 
performance  rating  which  is  on  a ratio  scale.  The  only 
assumption  necessary  for  the  Friedman  test  is  that  the 
observations  are  independent  and  that  the  variables  under 
study  have  underlying  continuity  (37:31).  The  data  was 
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placed  in  a two-way  table  having  N rows  and  k columns 
(37:166).  The  rows  represent  the  type  of  troubleshooting 
aid  used  and  the  columns  represent  the  18  subjects  in  the 
groups.  Each  row  shows  the  rank  of  the  subject's  perform- 
ance, with  that  aid,  relative  to  the  other  17  subjects. 

The  Friedman  test  determines  if  the  columns  of  ranks  came 
from  the  same  population  by  determining  whether  the  rank 

totals  differ  significantly  (37:168).  To  make  this  test, 

2 

the  value  of  a statistic  denoted  as  xr  roust  be  computed. 

2 

Since  it  can  be  shown  that  xr  is  distributed  approximately 
as  chi  square,  the  chi  square  tables  were  used  for  tests 
of  significance  (37:168).  The  level  of  significance  (a) 
was  set  at  .05. 

These  are  the  steps  which  were  used  in  the  Friedman 
two-way  analysis  of  variance  by  ranks  (37:171): 

1.  The  performance  scores  for  the  subjects  from 
the  experience  level  being  considered  were  cast  in  a two- 
way  table  having  18  columns  and  2 rows  for  the  Keesler 
subjects,  or  3 rows  for  the  MAC  subjects. 

2.  The  performance  ratings  in  each  row  were  ranked 
from  1 to  18. 

3.  The  ranks  in  each  column  were  then  summed. 

4.  The  test  statistic  was  computed  using  the 
following  formula: 


k 

2 _ 12  V 

r Nk  (k+1)  Z-/ 

j = l 


(Rj)2  - 3N (k+1 ) 
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where 


N = number  of  rows, 
k = number  of  columns, 

R . = sum  of  ranks  in  j*"*1  column,  and 

V 

k 

= the  sum  of  the  squares  of  the  sums  of 
3=1  ranks  over  all  k. 

5.  The  associated  probability  of  occurrence  under 

9 

H0  associated  with  the  computed  value  of  xr  was  determined 
by  reference  to  the  chi-square  distribution  with  df  - k- 1. 

6.  If  the  probability  given  by  the  chi-square 
distribution  is  equal  to  or  less  than  a (.05),  then  was 
rejected. 

Criterion  Test 

The  objective  of  Research  Proposition  No.  1 was  to 
show  that  there  are  individuals  who  possess  native  trouble- 
shooting ability.  The  Friedman  test  was  used  to  determine 
if  there  were  significant  differences  among  the  rankings  of 
the  subjects  in  each  sample  group.  In  order  to  infer  that 
certain  subjects  possessed  native  troubleshooting  ability, 
a criterion  test  was  also  needed.  The  criterion  upon  which 
an  inferrence  of  native  troubleshooting  ability  could  be 
made  was  that  a statistically  significant  difference  among 
the  mean  ranks  of  performance  of  the  subjects  must  be 
shown  for  all  three  sample  groups. 
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Research  Proposition  No.  2 


Data  Collection 

The  troubleshooting  performance  data,  as  described 
for  Research  Proposition  No.  1,  was  also  used  in  the  analy- 
sis of  Research  Proposition  No.  2. 


Description  of  Variables 

The  variables  of  primary  concern  here  are  perform- 
ance and  experience  level.  Since  a single  performance  mea- 
sure was  needed  for  each  subject,  individual  performance 
indices  for  each  type  of  troubleshooting  aid  were  averaged. 
Thus,  an  average  performance  index  was  formulated  as 
follows: 


n 


PI 


AP  = 


i=l 


n 


where  AP  = average  performance  index, 

PI  = performance  index  previously  computed  for  the 
£th  troubleshooting  aid,  and 

n = number  of  types  of  troubleshooting  aids  with 
which  the  subject  was  evaluated  (2  or  3). 


This  average  performance  index  provided  a single  perform- 
ance measure  for  each  subject. 

Three  different  experience  levels  of  the  328X4 
AFSC  were  sampled  for  the  AFHRL  project.  The  first  group 
consisted  of  recent  graduates  of  the  Avionics  Inertial  and 
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Radar  Navigation  Systems  Specialist  Technical  Training 
Course  at  KTTC.  These  subjects  had  no  field  experience. 

The  second  group  consisted  of  MAC  personnel  who  had  six 
months  or  less  experience  in  the  career  field.  The  third 
group  was  composed  of  MAC  personnel  with  more  than  six 
months  experience. 

Design  to  Test  Research 
Proposition  No.  2 

The  objective  of  the  second  research  proposition 
was  to  determine  if  a significant  difference  exists  among 
the  troubleshooting  performance  of  technicians  at  the  three 
different  experience  levels.  The  Kruskal-Wallis  one-way 
analysis  of  variance  by  ranks  was  used  to  determine  if 
differences  existed  between  the  experience  levels.  The 
Kruskal-Wallis  technique  is  a nonparametric  test  used  for 
deciding  whether  two  or  more  samples  are  from  different 
populations  (37:184).  The  null  hypothesis  {E Q)  for  the 
test  was  that  there  is  no  difference  between  the  average 
performance  indices  of  technicians  with  no  experience, 
technicians  with  six  months  or  less  experience,  and  tech- 
nicians with  more  than  six  months  experience.  The  alter- 
nate hypothesis  (H was  that  there  is  a difference  in  the 
average  performance  indices  of  the  three  groups. 

Kruskal-Wallis  One-Way  Analysis  of  Variance  by 
Ranks.  The  Kruskal-Wallis  test  requires  that  the  variable 
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under  study  be  in  at  least  an  ordinal  scale  (37:184).  The 
sample  data  which  was  used  in  this  analysis  is  on  a ratio 
scale 

For  the  analysis,  the  average  performance  indices 
for  all  54  subjects  were  ranked  in  a single  series  (37:185). 
After  the  indices  had  been  ranked,  the  sum  of  the  ranks  for 
each  experience  level  was  computed.  The  Kruskal-Wallis 
test  determines  whether  these  sums  of  ranks  are  signifi- 
cantly different. 

The  steps  which  were  used  in  the  Kruskal-Wallis 
test  are  summarized  as  follows  (27:192): 

1.  All  of  the  performance  indices  for  the  three 
sample  groups  were  ranked  in  a single  series,  with  ranks 
assigned  from  1 to  54. 

2.  The  sum  of  the  ranks  for  each  experience  level 
was  determined. 

3.  The  test  statistic,  H,  was  computed  as  follows: 


H 


12 

N (N+l) 


3 (N+l) 


where 


k = number  of  samples  (i.e.,  experience  levels), 

n . = number  of  cases  in  j*"*1  sample, 

3 

N = , the  number  of  cases  in  all  samples 

combined. 
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R . - sum  of  ranks  in  j sample,  and 


£ directs  one  to  sum  over  the  k samples. 


4.  It  can  be  shown  that  if  the  samples  actually 
are  from  the  same  population  or  identical  populations,  then 
H is  distributed  as  chi  square  with  df  = fc-1  (37:185). 
Therefore,  the  significance  of  a value  as  large  as  the 
observed  value  of  E was  assessed  by  reference  to  the  chi 
square  table  with  df  - k- 1. 

5.  The  level  of  significance  (a)  for  this  analysis 
was  set  at  . 05.  If  the  probability  associated  with  the 
observed  value  of  3 is  equal  to  or  less  than  a,  H Q was 
rejected. 

When  ties  occurred  between  one  or  more 
indices,  each  index  was  given  the  mean  of  the  ranks  for 
which  it  was  tied  (37:188).  Since  the  value  of  H is  some- 
what influenced  by  ties,  the  Kruskal-Wallis  test  allows  a 
correction  for  a large  number  of  ties.  To  correct  for 
ties,  H is  computed  as  shown  in  step  3 and  then  divided 
by  (37:188): 


N3-N 


where  T = t -t  (t  is  the  number  of  tied  observations  in 
a tied  group  of  scores) , 


».-•/  * ‘A •». 


N = number  of  observations  in  all  k samples 
together,  and 

^ T directs  one  to  sum  over  all  groups  of  ties. 

The  effect  of  correcting  for  ties  is  to  increase 
the  value  of  H and  make  the  results  more  significant  than 
if  uncorrected. 

Criterion  Test 

Research  Proposition  No.  2 attempts  to  determine 
if  significant  differences  exist  in  level  of  performance 
at  different  experience  levels.  The  analysis  was  performed 
at  a significance  level  of  .05.  If  E is  rejected  at  a = 
.05,  then  it  could  be  inferred  that  the  level  of  experience 
does  make  a significant  difference  in  performance  and 
further  analysis  of  the  data  must  consider  each  group 
separately. 


Research  Proposition  No.  3 

Data  Collection 

The  subjects  at  Keesler  were  available  to  partici- 
pate in  the  AFHRL  experiment  for  the  entire  day  for  a two- 
week  period.  Each  subject  was  scheduled  for  t-oubleshooting 
evaluation  for  only  one-half  of  each  day  (30:30).  The 
half-day  committed  to  this  evaluation  was  alternated  in  an 
attempt  to  remove  any  time-of-day  effect  of  a particular 
problem,  troubleshooting  aid  type,  or  subject.  The 
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one- ha If  day  not  involved  with  troubleshooting  was  used  to 
collect  ancillary  data,  such  as  information  on  the  indi- 
vidual’s background,  personality,  attitude,  and  aptitude. 
This  research  is  primarily  concerned  with  the  personality 


data. 


With  the  MAC  sample  groups,  ancillary  data  was 


obtained  at  any  time  the  subject  was  available  and  was  not 
required  for  troubleshooting  evaluation  activities  (30:57). 
This  procedure  was  necessary  since  the  MAC  subjects  were 
made  available  for  only  four  hours  each  day.  Identical 
ancillary  data  to  that  collected  with  the  Keesler  subjects 
was  obtained. 

The  personality  test  which  was  administered  during 
the  AFHRL  project  was  a 476-item  Adjustment  Prediction 
Inventory  developed  by  Norman  R.  Potter  (30:31).  Imbedded 
in  this  inventory  is  the  Gordon  Personal  Profile  and  the 
Gordon  Personal  Inventory  Instruments.  This  research  was 
concerned  with  those  personality  traits  measured  by  the  two 
Gordon  instruments. 


Personality  Assessment  Instruments 

The  Gordon  Personal  Profile  measures  four  separate 
aspects  of  personality  which  are  significant  in  the  daily 
functioning  of  the  normal  person  (16:3).  The  four  traits 
measured  are  Ascendancy,  Responsibility,  Emotional  Sta- 


bility, and  Sociability.  These  four  traits  are  relatively 


independent  and  psychologically  meaningful  traits  which 
have  been  found  to  be  important  in  determining  the  adjust- 
ment and  effectiveness  of  an  individual  in  many  social, 
educational,  and  industrial  situations  (16:3). 

The  Profile  is  made  up  of  eighteen  sets  of  four 
descriptive  phrases.  Each  set  of  four  phrases  is  known 
as  a "tetrad"  (16:3).  Each  of  the  four  personality  traits 
is  represented  by  one  of  the  descriptive  phrases  in  each 
tetrad.  The  respondent  is  asked  to  mark  one  item  in  each 
tetrad  as  being  most  like  himself  and  one  as  being  least 
like  himself. 

The  Gordon  Personal  Inventory  measures  four  addi- 
tional traits  and  is  used  as  a companion  instrument  with 
the  Personal  Profile  (15:3).  The  traits  measured  are 
Cautiousness,  Original  Thinking,  Personal  Relations , and 
Vigor.  Both  the  Inventory  and  the  Profile  may  be  used 
with  high  school,  college,  industrial,  or  general  adult 
groups.  The  format  of  the  Inventory  is  the  same  as  that 
of  the  Profile  except  that  the  Inventory  consists  of 
twenty  sets  of  tetrads.  Both  instruments  make  use  of  the 
forced-choice  technique;  that  is,  the  respondent  must  mark 
the  phrase  most  like  and  least  like  himself  in  each  tetrad. 
Since  the  respondent  can  not  respond  favorably  to  all  four 
items,  the  forced-choice  approach  is  believed  to  be  less 
susceptible  to  distortion  by  individuals  who  are  motivated 
to  make  a good  impression  (15:3). 
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Validity  and  Reliability 
of  Instruments 


Both  the  Personal  Profile  and  the  Personal  Inven- 
tory are  accepted  personality  assessment  instruments  whose 
validity  and  reliability  have  been  established  through 
various  studies  (15:10;  16:10).  These  instruments  have 
achieved  wide  usage  in  industrial  personnel  selection 
(16:15).  The  Gordon  instruments  were  deemed  appropriate 
for  the  AFHRL  project  because  they  are  easily  administered, 
scored,  and  interpreted  (31).  The  tests  are  virtually 
self-administering  and  can  be  completed  in  10  to  15 
minutes.  Additionally,  the  lack  of  correlation  with  intel- 
ligence and  aptitude  measures  allows  the  Gordon  instru- 
ments to  make  an  independent  contribution  in  predicting 
performance  criteria  (16:15). 

Description  of  Variables 

Research  Proposition  No.  3 dealt  with  correlating 
troubleshooting  performance  with  the  personality  traits 
measured  by  the  Gordon  instruments  in  an  attempt  to 
identify  those  traits  which  can  be  used  as  predictors  of 
success  in  troubleshooting  tasks.  The  performance  variable 
consisted  of  an  average  performance  index  for  each  subject. 
The  procedure  was  to  take  the  performance  ratings  com- 
puted for  each  type  of  troubleshooting  aid  for  each  subject 
for  the  first  proposition  and  average  those  ratings.  The 
result  was  a single  performance  measure  for  each  subject. 
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These  performance  measures  were  then  correlated  with  the 
scores  obtained  from  the  Gordon  personality  tests.  • 
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A total  of  eight  personality  traits  were  measured 
by  the  Gordon  instruments.  The  following  is  a description 
of  these  eight  traits  in  terms  of  the  interpretation  of 
high  and  low  scores  (15:3;  16:3). 

1.  Ascendancy.  Those  individuals  who  are  verbally 
ascendant,  adopt  an  active  role  in  the  group,  are  self- 
assured  and  assertive  in  relationships  with  others,  and 
tend  to  make  independent  decisions,  score  high  on  this 
scale.  Those  who  play  a passive  role  in  the  group,  listen 
rather  than  talk,  lack  self-confidence,  let  others  take 
the  lead,  and  tend  to  be  overly  dependent  on  others  for 
advice,  normally  make  low  scores. 

2.  Responsibility.  The  high  scores  on  this  scale 
are  those  individuals  who  are  persevering  and  determined, 
are  able  to  stick  to  any  job  assigned  them,  and  can  be 
relied  on.  Individuals  who  are  unable  to  stick  to  tasks 
that  do  not  interest  them,  and  who  tend  to  be  irresponsible 
usually  score  low. 

3.  Emotional  Stability.  High  scores  on  this  scale 
are  generally  made  by  individuals  who  are  well-balanced, 
emotionally  stable,  and  relatively  free  from  anxieties  and 
nervous  tension.  Low  scores  are  associated  with  excessive 
anxiety,  hypersensitivity,  nervousness,  and  low  frustration 
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tolerance.  Generally  a very  low  score  reflects  poor  emo- 
tional balance. 

4.  Sociability.  High  scores  are  made  by  indi- 
viduals who  like  to  be  with  and  work  with  people,  and  who 
are  gregarious  and  sociable.  Low  scores  reflect  a lack  of 
gregariousness,  a general  restriction  in  social  contacts, 
and,  in  the  extreme,  an  actual  avoidance  of  social  rela- 
tionships. 

5.  Cautiousness.  Individuals  who  are  highly 
cautious,  who  consider  matters  very  carefully  before  making 
decisions,  and  do  not  like  to  take  chances  or  run  risks, 
score  high  on  this  scale.  Those  who  are  impulsive,  act 

on  the  spur  of  the  moment,  make  snap  decisions,  enjoy 
taking  chances,  and  seek  excitement,  score  low. 

6.  Original  Thinking.  High  scoring  individuals 
like  to  work  on  difficult  problems,  are  intellectually 
curious,  enjoy  thought-provoking  questions  and  discussions, 
and  like  to  think  about  new  ideas.  Low  scoring  individuals 
dislike  working  on  difficult  or  complicated  problems,  do 
not  care  about  acquiring  knowledge,  and  are  not  interested 
in  thought-provoking  questions  or  discussions. 

7.  Personal  Relations.  High  scores  are  made  by 
those  individuals  who  have  great  faith  and  trust  in  people, 
and  are  tolerant,  patient,  and  understanding.  Low  scores 
reflect  a lack  of  trust  or  confidence  in  people,  ,and  a 
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tendency  to  be  critical  of  others  and  to  become  annoyed  or 
irritated  by  what  others  do. 

8.  Vigor . High  scores  on  this  scale  characterize 
individuals  who  are  vigorous  and  energetic,  who  like  to 
work  and  move  rapidly,  and  who  are  able  to  accomplish  more 
than  the  average  person.  Low  scores  are  associated  with 
low  vitality  or  energy  level,  a preference  for  setting  a 
slow  pace,  and  a tendency  to  tire  easily  and  be  below 
average  in  terms  of  sheer  output  or  productivity. 

Design  to  Test  Research 
Proposition  No.  3 

The  third  research  proposition,  that  a relationship 
exists  between  troubleshooting  performance  and  personality 
characteristics,  was  tested  by  means  of  eight  separate 
hypothesis  tests;  one  test  for  each  of  the  eight  traits 
measured  by  the  Gordon  instruments.  The  statistical  tech- 
nique used  was  the  nonparametric  Spearman  rank  correlation 
coefficient.  The  null  hypothesis  (H Q)  for  each  test  was 
that  the  performance  ratings  and  the  personality  scores  are 
not  related.  The  alternate  hypothesis  [H ^ ) was  that  the 
two  variables  are  related. 

Spearman  Rank  Order  Correlation  Coefficient.  The 
Spearman  test  is  a measure  of  association  which  requires 
that  the  variables  be  in  at  least  an  ordinal  scale  (37: 
202).  The  performance  variable  for  this  research  was  on  a 


ratio  scale  and  the  personality  test  scores  were  on  an 
ordinal  scale. 


For  the  Spearman  test  the  subjects  were  ranked 
in  a two-ordered  series  according  to  performance  index 
and  personality  trait  score  (37:202).  The  steps  necessary 
in  the  use  of  the  Spearman  rank  correlation  coefficient 
for  each  group  are  as  follows  (37:212): 

1.  The  performance  ratings  and  personality  scores 
were  each  ranked  from  1 to  54. 

2.  Each  of  the  subjects  was  listed  with  his  rank 
on  the  performance  variable  and  the  personality  variable 
next  to  his  entry. 

3.  The  value  of  was  determined  by  subtracting 
the  personality  variable  rank  from  the  performance  vari- 
able rank.  This  value  was  squared  and  summed  to  obtain 


4.  The  test  statistic,  r , was  computed  as  follows: 

s 


T 


whether  the  observed  value  of  r indicates  an  association 

s 

between  the  variables  in  the  population.  The  significance 
of  a value  as  large  as  the  computed  value  of  r may  be 
determined  by  computing  the  t associated  with  the  ?s  value. 
The  significance  of  that  value  of  t can  be  determined  by 
referring  to  the  table  of  critical  values  of  t.  The  value 
of  t is  computed  by  the  formula: 


t = r 


where 


N = the  total  number  of  subjects,  and 


= the  computed  correlation  coefficient. 


The  value  of  t is  distributed  as  Student's  t with  df  = 


When  tied  scores  occurred,  each  of  them  was 
assigned  the  average  of  the  ranks  for  which  they  were  tied. 
If  the  proportion  of  ties  in  the  Spearman  test  is  not 
large,  their  effect  on  is  negligible  and  the  formula 
described  above  may  still  be  used  (37:206).  However,  when 
the  proportion  of  ties  is  large,  a correlation  factor  must 
be  used  in  the  computation  of  r . 

The  correlation  factor  is  calculated  by: 


t3~t 


*•»  <1  V *.  **.  . •’«#  ' »»»* 


• *•* 


where 


t = the  number  of  tied  observations  at  a 
given  rank. 


When  a considerable  number  of  ties  were  present, 

the  following  formula  was  used  to  compute  r : 
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r = 
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Research  Proposition  No.  3 was  concerned  with  the 
identification  of  one  or  more  personality  variables  which 
are  related  to  troubleshooting  performance.  The  signifi- 
cance level  for  testing  the  eight  hypotheses  for  each  group 
was  set  at  .05.  In  order  to  support  the  research  proposi- 
tion, the  criterion  test  was  that  a minimum  of  one  personal- 
ity variable  needed  to  be  found  to  be  significantly  related 
to  troubleshooting  performance. 


Research  Proposition  No.  4 


Data  Collection 


Since  1959,  the  Airman  Qualifying  Examination  (AQE) 


screening  and  for  the  classification  of  basic  trainees  for 
technical  training  (41:1).^  The  AQE  consists  of  about  200 
aptitude  items  and  is  designed  to  evaluate  airmen  in  terms 
of  four  aptitude  areas  considered  important  for  success  in 
certain  AF  schools  and  airman  career  fields  (42:12-1). 
Minimum  scores  in  these  aptitude  areas  are  considered 
desirable  for  entry  into  certain  career  fields  and  for 
admission  to  certain  AF  technical  training  courses.  Air- 
men who  receive  at  least  the  minimum  scores  and  are  other- 
wise qualified  are  considered  to  have  reasonable  expecta- 
tions of  being  successful  in  the  training  or  duty  for  which 
the  minimum  score  is  specified  (42:12-1).  Although  a 
number  of  factors  enter  into  an  airman's  assignment,  the 
most  important  is  his  score  on  the  aptitude  area  appropri- 
ate to  his  broad  career  field  (41:1).  The  AQE  is  composed 
of  a series  of  subtests  which  are  combined  into  four  apti- 
tude clusters:  Mechanical,  Administrative,  Electronics, 

and  General. 

The  AQE  is  composed  of  a total  of  10  subtests. 

Each  of  the  four  aptitude  clusters  includes  one  or  more  of 
the  subtests  and  each  subtest  may  be  included  in  more  than 
one  aptitude  cluster.  The  following  is  a list  of  the 
aptitude  clusters  and  the  subtests  in  each  cluster  (44:2). 

^The  AQE  has  since  been  changed  to  the  Airman 
Classification  Test  (ACT);  however,  the  ACT  aptitude  areas 
are  comparable  in  magnitude  and  content  to  AQE  scores  on 
the  same  areas  (42:12-1). 
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Mechanical 


1.  General  Mechanics 

2.  Hidden  Figures 

3.  Mechanical  Principles 

4.  Shop  Practices 

Administrative 


1.  Arithmetic  Computation 

2.  Arithmetic  Reasoning 

3.  Word  Knowledge 

Electronics 


1.  Arithmetic  Reasoning 

2.  Data  Interpretation 

3.  Electrical  Information 

4.  Pattern  Comprehension 

General 


1.  Arithmetic  Reasoning 

2.  Hidden  Figures 

3.  Word  Knowledge 

The  AQE  scores  for  the  subjects  in  the  AFHRL  study 
were  taken  from  the  personnel  records  of  each  subject. 
Entry  into  the  328X4  AFSC  career  field,  and  therefore  the 
KTTC  training  course,  requires  a score  of  at  least  80  in 
the  Electronics  aptitude  cluster  of  the  AQE  (43:A16-84). 
The  Electronics  score  is  the  primary  consideration  for 
entry  into  the  career  field;  however,  for  the  sake  of 
research,  this  analysis  considered  all  four  scores. 


Description  of  Variables 

Research  Proposition  No.  4 was  concerned  with  cor- 
relating troubleshooting  performance  and  aptitude  mea- 
sured by  the  AQE.  The  performance  variable  consisted  of  an 
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average  performance  index  for  each  subject  as  described  in 
Research  Proposition  No.  2.  The  aptitude  variables  con- 
sisted of  the  scores  from  each  AQE  aptitude  cluster  for 
each  subject.  What  follows  is  a description  of  the  apti- 
tude variables  in  terms  of  what  aspects  of  aptitude  each 
of  the  AQE  subtests  measures  (44:5). 

1.  Arithmetic  Computation.  This  subtest  consists 
of  simple  arithmetic  items  involving  addition,  subtraction, 
multiplication,  and  division  of  whole  numbers.  It  is 
administered  as  a speed  test  and  is  designed  to  measure 
the  ability  to  manipulate  numbers  rapidly  and  accurately. 

2.  Arithmetic  Reasoning.  This  subtest  evaluates 
the  examinee's  ability  to  think  through  mathematical 

\ 

problems  presented  in  verbal  form.  It  involves  the  dis- 
covery and  application  of  the  general  mathematical  princi- 
ples required  to  arrive  at  a correct  solution  to  each 
problem,  as  well  as  performance  of  the  necessary  calcula- 
tion to  attain  that  solution. 

3.  Data  Interpretation.  This  subtest  is  designed 
to  measure  the  ability  to  draw  conclusions  or  make  inter- 
pretations from  data  presented  in  the  form  of  graphs, 

1 

charts,  and  tables. 

4.  Electrical  Information.  The  ability  to  apply 
previously  acquired  knowledge  in  the  areas  of  electricity 
and  electronics  toward  the  solutions  of  practical  problems 
is  measured  by  this  subtest. 
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5.  General  Mechanics.  This  subtest  consists  of 
verbal  items  relating  to  the  understanding  and  application 
of  basic  techniques  required  for  the  troubleshooting  and 
repair  of  various  mechanical  devices. 

6.  Hidden  Figures.  This  subtest  requires  the 
examinee  to  determine  which  one  of  the  five  simple  line 
drawings  is  contained  in  a more  complex  arrangement  of 
geometric  figures. 

7.  Mechanical  Principles.  In  this  subtest,  the 
examinee  is  required  to  determine  from  pictures  of  mechani- 
cal devices  their  operating  characteristics. 

8.  Pattern  Comprehension.  This  subtest  involves 
visualizing  the  folding  of  flat  patterns  into  three- 
dimensional  objects  and  subsequently  determining  the  loca- 
tion of  specific  points  which  are  common  to  both  the  pat- 
tern and  the  solid  figure. 

9.  Shop  Practices.  This  is  a pictorial  subtest 
which  requires  the  examinee  to  identify  pictured  tools  and 
determine  their  proper  use  in  a specific  situation  or  the 
selection  of  the  proper  tool  for  use  in  a given  task. 

10.  Word  Knowledge.  This  is  a subtest  of  verbal 
ability  involving  the  definition  of  words.  It  is  a vocabu- 
lary test  of  nontechnical  terms. 
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Design  to  Test  Research 
Proposition  No.  4 

The  fourth  research  proposition,  that  there  is  a 
direct  relationship  between  troubleshooting  performance 
and  aptitude,  was  tested  through  four  separate  hypotheses 
tests;  one  test  for  each  of  the  four  aptitude  scores  from 
the  AQE . The  statistical  technique  used  was  the  nonpara- 
metric  Spearman  rank  correlation  coefficient.  The  Spearman 
test  was  performed  as  described  in  Research  Proposition 
No.  3.  The  null  hypothesis  (H  ) for  each  test  was  that  the 
performance  indices  and  the  aptitude  scores  are  not  related. 

The  alternate  hypothesis  (tf  ) was  that  the  two  variables 
are  related.  The  significance  level  for  testing  the  four 
hypotheses  for  each  group  was  set  at  .05. 

Criterion  Test 

In  order  to  support  the  research  proposition,  the 
criterion  test  was  that  at  least  one  AQE  aptitude  cluster 
needed  to  be  found  to  be  significantly  related  to  trouble- 
shooting performance. 

Summary  List  of  Assumptions 

1.  The  maintenance  level  of  troubleshooting  main- 
tenance tasks  is  not  significant. 

2.  The  observations  are  independent,  and  the  vari- 
ables under  study  have  underlying  continuity. 


r — ^ 

3.  The  standard  times  derived  for  replacement  of 
incorrect  parts  and  components  are  representative  times. 

Summary  List  of  Limitations 

1.  The  researchers  had  no  control  over  experiment 
design  or  data  collection.  The  data  was  collected  in  a 
previous  research  project  which  had  different,  though  not 
conflicting,  objectives. 

2.  The  performance  criteria  computations  used  in 
this  research  may  not  be  appropriate  for  all  troubleshoot- 
ing situations. 

3.  The  findings  of  this  research  can  not  be 
generalized  outside  the  population  of  interest. 


CHAPTER  IV 


ANALYSIS  AND  DISCUSSION 

This  chapter  contains  the  statistical  analysis 
and  discussion  of  each  Research  Proposition.  Discussion 
of  the  analysis  of  each  proposition  is  included  since  the 
results  were  not  as  originally  predicted.  In  some  areas, 
additional  analysis  was  conducted  in  an  effort  to  gain 
greater  insight  into  what  factors  might  have  caused  the 
results  to  differ  from  what  was  predicted. 

Consistency  of  Individual  Performance 
Using  Various  Types  of  Aids 

The  first  research  proposition  was  that  "Individ- 
uals who  perform  well  using  one  type  of  troubleshooting 
aid,  tend  to  perform  well  using  the  other  types  of  trouble- 
shooting aids."  The  objective  of  this  proposition  was  to 
demonstrate  that  some  of  the  subjects  possessed  a higher 
level  of  native  troubleshooting  ability  than  the  others. 

To  determine  support  or  nonsupport  for  the  proposition. 
Hypotheses  1 through  3 were  tested  using  the  Friedman  two- 
way  analysis  of  variance  test.  Thus,  the  following  general 
hypothesis  was  tested  for  each  experience- level  subject 
group. 


H : There  is  no  difference  in  the  mean  ranks  of  per- 

formance of  subjects  using  various  troubleshoot- 
ing aids. 


H . : There  is  a difference  in  the  mean  ranks  of 

performance  of  subjects  using  various  trouble- 
shooting aids. 

If  the  mean  ranks  of  the  performance  of  the  subjects,  across 
the  various  types  of  troubleshooting  aids  used,  is  signifi- 
cantly different,  the  null  hypothesis,  E Q , would  be  rejected 
by  the  Friedman  test.  The  rejection  of  EQ  would  enable  one 
to  conclude  that  the  subjects'  performance  rankings  were  con- 
sistently similar,  regardless  of  the  type  of  aid  used 
(i.e.,  subjects  who  perform  well  using  one  type  of  trouble- 
shooting aid,  perform  well  using  the  other  types  of 
troubleshooting  aids) . It  could  then  be  inferred  that  the 
subjects  which  performed  well,  regardless  of  the  type  of 
aid  being  used,  possess  a higher  level  of  native  ability 
than  the  subjects  which  did  not  perform  as  well. 

On  the  other  hand,  if  the  subjects  performed  incon- 
sistently across  the  various  types  of  troubleshooting  aid 
used  (i.e.,  did  quite  well  when  using  one  type  of  aid,  but 
very  poorly  using  another  type) , then  the  distribution  of 
ranks  in  each  column  (by  subject)  would  appear  to  be  a 
matter  of  chance  and  the  null  hypothesis  would  not  be 
rejected.  For  the  research  proposition  to  be  supported, 
the  null  hypothesis,  E q,  must  be  rejected  in  all  three 
hypothesis  tests. 

Appendix  A,  Tables  A-l,  A-2,  and  A-3  contain 
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experience-level  subject  groups.  The  rank  ordering  within 

. 2 

experience-level  groups  and  the  Friedman  xr  computations 
are  shown  in  Appendix  A,  Tables  A-4,  A-5,  and  A-6. 

As  indicated  in  Table  1A,  the  null  hypothesis  could 
not  be  rejected,  at  the  previously  established  .05  level  of 
significance,  for  any  of  the  three  experience-level  subject 
groups.  Examination  of  the  rank  ordering  of  performance 
(as  shown  in  Appendix  A,  Tables  A-4,  A-5,  and  A-6)  reveals 
large  differences  in  the  relative  performance  of  individ- 
uals when  using  different  troubleshooting  aids.  Some  indi- 
viduals performed  quite  well  using  one  type  of  troubleshoot- 
ing aid  but  quite  poorly  using  one,  or  more,  of  the  other 
types  of  aids.  Consequently,  the  research  proposition  was 
not  supported,  and  thus  it  could  not  be  inferred  from  the 
data  and  statistical  tests  that  some  of  the  subjects 
possessed  a higher  level  of  native  ability  than  the  others. 

Additional  Analysis  and  Discussion 

It  was  originally  predicted  that  these  three  hypo- 
thesis tests  would  support  the  research  proposition.  Since 
the  tests  did  not  support  the  research  proposition,  it  was 
decided  to  conduct  additional  analysis  of  the  data  to 
determine  if  native  ability  was  merely  being  masked  by 
certain  factors  or  if  it  actually  did  not  exist.  The 
authors  felt  that  several  conditions/factors  might  have 
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TABLE  1A 


CONSISTENCY  OF  RELATIVE  INDIVIDUAL  PERFORMANCE 
USING  VARIOUS  TYPES  OF  TROUBLESHOOTING  AIDS: 
RESULTS  OF  FRIEDMAN  TWO-WAY  ANALYSIS 
OF  VARIANCE  TEST 


Subject  Group 

N 

X* 

P 

Zero  Experience 

18 

21.123 

.20 

A 

A 

.30 

£ 6 Months  Experience 

18 

15.450 

.50 

< p < 

.70 

> 6 Months  Experience 

18 

25.269 

. 05 

A 

A 

.10 

2 

Critical  Value  x = 

27.587, 

a = .05, 

df  = 

17. 

masked  the  existence  of  native  ability,  and,  consequently, 
additional  analysis  was  conducted  in  these  areas. 

1.  Types  of  troubleshooting  aid  had  an  overpower- 
ing effect.  A possible  explanation  of  why  native  ability 
was  not  evident  is  that  the  type  of  troubleshooting  aid 
being  used  had  a significant  effect  on  the  subjects'  over- 
all performance.  In  order  to  determine  the  extent  of  the 
effect  of  type  of  aid,  the  Friedman  test  was  again  used. 

For  this  analysis,  the  rows  and  columns  were  reversed 
(i.e.,  the  subjects  were  now  placed  in  the  rows  and  the 
types  of  aids  in  the  columns) . 

The  rank  ordering  of  the  aids  within  each  subject 

2 

group  and  the  computation  of  the  Friedman  xr  statistics  are 
shown  in  Appendix  A,  Tables  A-7,  A-8,  and  A-9.  The  follow- 
ing general  hypothesis  test  was  conducted  for  each 
experience- level  subject  group: 
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H : The  type  of  troubleshooting  aid  being  used  had 

0 no  effect  on  troubleshooting  performance. 

E : Troubleshooting  performance  was  dependent  upon 

the  type  of  troubleshooting  aid  being  used. 

As  Table  IB  indicates,  the  null  hypothesis  was 
rejected  for  both  the  zero-experience  group  and  the  six- 
months-or-less-experience  group.  Thus,  there  is  evidence 
that  the  performance  of  the  subjects  in  the  first  two 
groups  was  dependent  on  the  type  of  aid  being  used.  This 
dependency,  however,  did  not  hold  true  for  the  more-than- 
six-months- experience  group. 

TABLE  IB 

EFFECT  OF  TYPE  OF  AID  ON  TROUBLESHOOTING  PERFORMANCE: 
RESULTS  OF  FRIEDMAN  TWO-WAY  ANALYSIS 
OF  VARIANCE  TEST 


Subject  Group 

N 

2 

xr 

P 

Zero  Experience 

2 

5 . 555a 

.01 

< P 

< 

.02 

£ 6 Months  Experience 

3 

21. 333b 

P 

< 

.01 

> 6 Months  Experience 

3 

.lllb 

. 90 

< P 

< 

. 95 

aCritical  Value 

2 

X * 

3.841, 

a = .05, 

df  = 

1. 

^Critical  Value 

2 

X = 

5. 991, 

a = .05, 

df  = 

1. 

These  findings  seem  to  coincide  with  the  results  of 
the  previous  Friedman  tests.  An  examination  of  Tables  A-4, 
A- 5,  and  A- 6 in  Appendix  A reveals  that  the  more-than- 
six-months-experience  group  came  closest  to  demonstrating 
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a consistency  of  performance  across  all  types  of  aids  (i.e., 
rejecting  E Q) , which  also  indicates  that  the  performance  of 
this  group  was  not  strongly  dependent  upon  the  type  of  aid 
being  used. 

2.  The  standard  times  used  for  replacement  of 
spare  parts  could  have  masked  differences  in  ability.  The 
performance  indices  initially  computed  for  each  subject 
included  a "standard"  time  for  replacing  any  spare  parts 
which  the  subject  would  have  erroneously  replaced.  If  a 
subject  actually  did  possess  native  ability,  then  the  possi- 
bility exists  that  he  could  have  replaced  any  parts  in  less 
than  the  standard  time.  Thus  he  would  have  been  penalized 
unfairly  by  including  this  time  in  his  performance  index. 

In  order  to  determine  if  the  spare-part  time  was 
a significant  factor,  performance  indices  were  computed 
for  the  subjects  in  each  group  with  the  spare-part  time 
excluded  from  the  total  time.  The  computations  of  the 
new  performance  indices  are  shown  in  Appendix  A,  Tables 
A- 10,  A- 11,  and  A- 12.  The  Friedman  test  was  then  conducted 
using  the  recomputed  performance  indices.  Appendix  A, 

Tables  A-13,  A-14,  and  A-15,  contain  the  rank  ordering 
within  each  subject  group  and  the  computations  for  the 
Friedman  y2. 

As  indicated  in  Table  1C,  the  null  hypothesis  of 
no  difference  in  the  mean  ranks  of  subjects'  performance 
could  not  be  rejected  for  the  zero-experience  group  or  the 
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TABLE  1C 


CONSISTENCY  OF  RELATIVE  INDIVIDUAL  PERFORMANCE  USING 
VARIOUS  TYPES  OF  TROUBLESHOOTING  AIDS  WITH 
SPARE- PART  TIME  NOT  CONSIDERED:  RESULTS 
OF  FRIEDMAN  TWO-WAY  ANALYSIS  OF 
VARIANCE  TEST 


Subject  Group 

N 

2 

xr 

P 

Zero  Experience 

18 

20.281 

• 

K> 

O 

A 

A 

.30 

< 6 Months  Experience 

18 

17.609 

.30 

< P < 

.50 

> 6 Months  Experience 

18 

27.725* 

. 02 

A 

'a 

A 

in 

o 

* 

2 

cal  Value  x * 

27. 587, 

p 

ii 

• 

o 

Ul 

df  = 

17. 

. i a*  ^ t a * 

. 05. 

six-roonths-or- lass-exp*  - i •?  ice  group.  But  for  the  more-than- 
six-months-experience  group,  ti  null  hypothesis  was 
rejected  at  the  .05  level  of  significance.  Thus,  there  is 
evidence  that  when  spare-part  time  is  not  considered,  sub- 
jects with  more  than  six  months  experience  perform  con- 
sistently across  all  three  types  of  troubleshooting  aids. 

Due  to  the  nature  of  the  types  of  troubleshooting 
aids,  one  might  think  that  the  exclusion  of  spare-part  time 
would  have  been  most  significant  in  the  subjects'  perform- 
ance using  the  T.O.  (i.e.,  that  more  spare-parts  would  have 
been  replaced  erroneously  while  using  the  T.O.).  However, 
a comparison  of  the  rank  ordering  of  the  more- than- six- 
months-  experience  subjects  when  spare-part  time  was 
included  in  the  performance  indices  (Appendix  A,  Table  A- 6) 
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and  the  rank  ordering  when  spare-part  time  was  excluded 
(Appendix  A,  Table  A-15),  does  not  reveal  any  specific 
pattern  in  changing  of  ranks.  That  is,  the  rank  changes 
due  to  exclusion  of  spare-part  time  do  not  appear  to  be 
confined  to  a particular  type  of  troubleshooting  aid,  but 
instead  affect  each  of  the  three  aids. 

3.  There  was  not  sufficient  difference  in  native 
ability  within  the  subject  groups.  The  time  and  cost 
involved  in  collecting  the  performance  data  for  the  original 
AFHRL  project  necessitated  a small  sample  size.  Although 
the  samples  were  believed  to  be  representative,  the  possi- 
bility exists  that  within  a sample  of  only  18  subjects,  the 
difference  in  the  ability  of  the  subjects  was  not  sufficient 
for  native  ability  to  be  revealed  in  the  statistical  tests. 

The  lack  of  sufficient  differences  in  ability 
could  also  be  attributed  to  the  fact  that  all  of  the  sub- 
jects had  already  been  screened  by  a selection  process 
based  on  aptitude  test  scores  in  the  Electronics  aptitude 
cluster  (a  minimum  score  of  80  is  required  for  selection 
for  the  technical  training  course  and  subsequent  entry  into 
the  career  field).  Since  there  was  no  way  to  test  either 
of  these  two  possibilities  without  actually  conducting 
additional  evaluations,  they  are  merely  stated  as  possible 
explanations. 


The  existence  of  native  ability  might  have  been 


masked  by  not  considering  the  maintenance  level  of  test 


problems . In  describing  the  testing  procedures  used  in 
collecting  the  data  for  the  AFHRL  project  (Chapter  III, 
pps.  48-49,51),  it  was  noted  that  the  test  problems  which 
were  presented  to  each  subject  group  were  divided  into  two 
maintenance  levels:  organizational  level  (flightline)  and 

intermediate  level  (shop) . Table  ID  shows  how  the  total 
number  of  problems  presented  to  each  subject  group  was 
broken  out  by  maintenance  level.  Since  it  is  possible 
that  individuals  vary  in  ability  to  troubleshoot  at  differ- 
ent maintenance  levels,  individual  performances  were 
reanalyzed  holding  the  level  of  maintenance  constant.  The 
subjects'  performance  indices  were  recomputed  for  each  of 
the  two  maintenance  levels.  Appendix  A,  Tables  A-16,  A-17, 
and  A- 18  contain  the  computations  of  performance  indices 
for  organizational  level  maintenance;  the  performance 
indices  for  the  intermediate- level  problems  as  shown  in 
Appendix  A,  Tables  A-22,  A-23,  and  A-24.  The  Friedman 
two-way  analysis  of  variance  by  ranks  test  was  again  used 
for  the  analysis  by  maintenance  level.  A separate  hypo- 
thesis test  was  conducted  for  each  of  the  two  maintenance 
levels  within  each  experience-level  subject  group. 

Analysis  of  Organizational-Level  Problems.  The 
rank  order  within  each  group  and  computation  of  the  Fried- 

9 

man  xr  statistics  for  the  organizational-level  problems 
are  contained  in  Appendix  A,  Tables  A-19,  A-20,  and  A-21. 
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TABLE  ID 


ORGANIZATIONAL  LEVEL  AND  INTERMEDIATE  LEVEL  PROBLEMS 
FOR  EACH  EXPERIENCE-LEVEL  GROUP 


Subject  Group 

Total 

Problems 

Organizational 

Level 

Intermediate 

Level 

Zero  Experience 

14 

4 

10 

_<  6 Months  Experience 

15 

4 

11 

> 6 Months  Experience 

15 

4 

11 

The  Following  general 

hypothesis 

was  tested  for 

each 

experience-level  subject  group: 

H Q:  There  is  no  difference  in  the  mean  ranks  of  sub- 

jects' performance  using  various  troubleshooting 
aids  at  the  organizational  maintenance  level. 

H i : There  is  a difference  in  the  mean  ranks  of  sub- 

jects' performance  using  various  troubleshooting 
aids  at  the  organizational  maintenance  level. 

The  results  of  the  hypotesis  test  for  each  subject 
group  are  shown  in  Table  IE.  The  null  hypothesis  could  not 
be  rejected,  at  the  .05  level  of  significance,  for  any  of 
the  three  experience- level  subject  groups.  An  examination 
of  the  rank  orderings  in  Appendix  A,  Tables  A-19,  A-20, 
and  A- 21,  reveals  the  same  conditions  as  in  previous  tests. 
That  is,  an  individual  who  performed  well  with  one  type  of 
aid  often  performed  poorly  with  another  type  of  aid.  Con- 
sequently, it  could  not  be  inferred  from  the  statistical 
test  that  some  of  the  subjects  possessed  a higher  level  of 
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TABLE  IE 


CONSISTENCY  OF  INDIVIDUAL  PERFORMANCE  USING  VARIOUS  TYPES 
OF  TROUBLESHOOTING  AIDS  AT  THE  ORGANIZATIONAL 
MAINTENANCE  LEVEL:  RESULTS  OF  FRIEDMAN 

TWO-WAY  ANALYSIS  OF  VARIANCE  TEST 


Subject  Group 

N 

2 

Xr 

V 

Zero  Experience 

18 

16.035 

.50 

A 

A 

.70 

< 6 Months  Experience 

18 

21.795 

.10 

< v < 

.20 

> 6 Months  Experience 

18 

6.833 

.99 

< P 

2 

Critical  Value  x = 

27.587, 

a = .05, 

df  = 

17. 

native  ability  than  did  others  at  the  organizational  main- 
tenance level. 

Analysis  of  Intermediate-Level  Problems.  Appendix 

A,  Tables  A- 25,  A-26,  and  A-27,  contain  the  rank  ordering 

2 

and  computations  of  the  Friedman  xr  statistics  for  the 
intermediate  maintenance  level.  The  following  general  hypo- 
thesis was  tested  for  each  experience-level  subject  group: 

H There  is  no  difference  in  the  mean  ranks  of  sub- 
jects' performance  .using  various  troubleshooting 
aids  at  the  intermediate  maintenance  level. 

H : There  is  a difference  in  the  mean  ranks  of  sub- 

jects' performance  using  various  troubleshooting 
aids  at  the  intermediate  maintenance  level. 

As  indicated  in  Table  IF,  the  null  hypothesis  could 
be  rejected,  at  the  .05  level  of  significance,  for  the  more- 
than  six-months-experience  group,  but  not  for  the  zero- 
experience  group  or  the  six-months-or-less-experience 
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TABLE  IF 


CONSISTENCY  OF  INDIVIDUAL  PERFORMANCE  USING  VARIOUS 
TYPES  OF  TROUBLESHOOTING  AIDS  AT  THE  INTERMEDIATE 
MAINTENANCE  LEVEL:  RESULTS  OF  THE  FRIEDMAN 

TWO-WAY  ANALYSIS  OF  VARIANCE  TEST 


Subject  Group 

N 

2 

xr 

P 

Zero  Experience 

18 

23.895 

.10 

A 

A 

.20 

< 6 Months  Experience 

18 

18.251 

.30 

A 

A 

.50 

> 6 Months  Experience 

18 

30.288* 

.02 

V 

n. 

V 

. 05 

2 

Critical  Value  xr  = 

27.587, 

in 

o 

• 

n 

a 

df  = 

17. 

♦Significant  at  a = 

. 05. 

group.  Thus,  there  is  evidence  that  for  subjects  with  more 
than  six  months  experience,  there  is  a significant  differ- 
ence in  the  mean  ranks  of  performance  when  troubleshooting 
at  the  intermediate  maintenance  level,  and  that  subjects  in 
this  group  maintained  consistent  relative  performance 
regardless  of  the  type  of  aid  used. 

These  findings  appear  to  be  consistent  with  the 
findings  of  previous  tests.  . In  the  initial  Friedman  test 
conducted  to  determine  the  consistency  of  performance  using 
various  aids,  it  was  the  more-than-six-months-experience 
group  which  came  closest  to  rejecting  the  null  hypothesis 
(reference  Table  1A)  and  thus  supporting  the  existence  of 
native  ability.  Additionally,  in  the  test  in  which  spare- 
part  time  was  excluded,  it  was  this  group  which  demonstrated 
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consistency  of  performance  across  all  three  types  of 
troubleshooting  aids  (reference  Table  1C) . Again,  in  the 
test  to  determine  the  effect  of  the  type  of  troubleshoot- 
ing aid  on  performance,  it  was  this  same  group  which 
showed  no  significant  differences  in  performance  related 
to  the  type  of  aid  (reference  Table  IB) . It  would  appear, 
then,  from  the  combination  of  these  results,  that  con- 
sistency of  performance  is  more  evident  for  subjects  at 
higher  experience  levels. 

Difference  in  Performance  Between 
Experience-Level  Groups 

Research  Proposition  No.  2 was  that  "A  significant 
difference  exists  between  the  troubleshooting  performances 
of  technicians  of  different  experience  levels."  To  deter- 
mine support  or  nonsupport  for  the  proposition,  Hypothesis 
No.  4 was  tested  using  the  Kruskal-Wallis  one-way  analysis 
of  variance  by  ranks  test.  The  following  hypothesis  was 
tested: 

H There  is  no  difference  in  the  troubleshooting 
performance  of  technicians  with  no  experience 
six  months  or  less  experience,  and  more  than 
six  months  experience. 

Hj : There  is  a difference  in  the  troubleshooting 

performance  of  technicians  with  no  experience 
six  months  or  less  experience,  and  more  than 
six  months  experience. 

The  performance  indices  used  in  this  test  were 
average  performance  indices  computed  by  averaging  the  sub- 
jects' performance  over  all  types  of  aids  used.  The 


computations  of  the  average  performance  indices  are  con- 
tained in  Appendix  B,  Tables  B-l,  B-2,  and  B-3.  The  rank 
ordering  of  the  subjects  and  the  computation  of  the 
Kruskal-Wallis  H are  contained  in  Appendix  B,  Table  B-4. 

As  indicated  in  Table  2A,  the  null  hypothesis  could 
not  be  rejected  at  the  previously  established  .05  level  of 
significance.  Therefore,  it  could  not  be  concluded  that 
the  performance  of  subjects  at  the  three  experience  levels 
was  significantly  different.  Table  2B  illustrates  the 
range  of  average  performance  indices  for  each  of  the 
experience- level  subject  groups. 

Although  this  test  indicates  otherwise,  it  would 
seem  that  there  would  be  a difference  in  performance  of 
different  experience- level  subjects  (i.e.,  that  more 
experienced  subjects  would  troubleshoot  more  effectively 
than  less  experienced  subjects).  However,  a further  con- 
sideration of  the  differences  in  the  types  of  troubleshoot- 
ing aids  helps  to  explain  why  this  test  indicates  no  dif- 
ference in  performance  at  different  experience  levels.  The 
basic  premise  behind  the  proposed  use  of  more  procedural- 
ized  aids  (FPTA  and  LTTA)  is  that  a technician  with  little 
or  no  experience,  when  using  proceduralized  aids,  can  per- 
form as  effectively  as  an  experienced  technician.  The 
results  of  this  test  appear  to  support  that  premise  to 
some  extent. 
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TABLE  2A 


* 


RESULTS  OF  TEST  FOR  DIFFERENCE  IN  PERFORMANCE  AMONG 
DIFFERENT  EXPERIENCE- LEVEL  SUBJECT  GROUPS 


2 

E 

Critical  x 

P 

3.850 

5.991 

.10  < p < .20 

a = .05,  df  = 2. 


TABLE  2B 

RANGE  OF  AVERAGE  PERFORMANCE  INDICES  FOR 


DIFFERENT 

EXPERIENCE- 

LEVEL  GROUPS 

Lowest  Avg. 

Median  Avg. 

Highest  Avg. 

Performance 

Performance 

Performance 

Subject  Group 

Index 

Index 

Index 

Zero  Experience 

33.170 

51.825 

74.681 

<_  6 Months  Experience 

20.002 

43.133 

78.233 

> 6 Months  Experience 

19.578 

34.542 

117.224 

Relationship  Between  Personality  Traits 
and  Troubleshooting  Performance 

The  third  research  proposition  was  that  "A  rela- 
tionship exists  between  specific  personality  traits  and 
demonstrated  troubleshooting  performance."  When  this 
proposition  was  originally  constructed,  it  was  believed 
that  statistical  tests  would  demonstrate  that  some  tech- 
nicians did  possess  native  troubleshooting  ability  and  also 
that  there  were  significant  differences  in  the  performance 
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of  technicians  at  different  experience  levels.  Since 
neither  of  these  beliefs  was  supported,  the  test  for  rela- 
tionship between  personality  traits  and  performance  could 
not  be  conducted  as  originally  intended  (i.e.,  by  conduct- 
ing a correlation  of  personality  traits  with  average  per- 
formance for  each  subject  group). 

The  authors  felt,  however,  that  some  insight  could 
still  be  gained  from  the  data  and,  therefore,  conducted 
some  additional  analysis.  First,  since  the  more- than- six- 
months-  experience-  level  subject  group  demonstrated  con- 
sistency of  performance  when  troubleshooting  at  the  inter- 
mediate maintenance  level,  a correlation  of  personality 
traits  with  the  average  performance  of  this  subject  group 
was  conducted.  Secondly,  a correlation  of  personality 
traits  with  all  the  subjects'  performance  with  each  type 
of  troubleshooting  aid  was  conducted. 

Analysis  of  the  More-Than-Six- 
Months-Experience-Level  Group 
at  the  Intermediate  Maintenance 
Level 

The  performance  indices  used  in  this  test  were 
computed  as  an  average  performance  index  across  all  three 
types  of  aids  used.  Appendix  C,  Table  C-l  contains  the 
computations  of  the  average  performance  indices.  The 
Spearman  rank  correlation  coefficient  test  was  used  to 
deteimine  if  a relationship  existed  between  the  average 
performance  indices  and  the  subjects'  scores  for  each  of 
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the  eight  personality  traits  measured.  The  subjects' 

personality  test  scores  are  contained  in  Appendix  C,  Table 

C-3.  Appendix  C,  Table  C-4,  contains  the  computations  of 

the  Spearman  r and  t statistics  for  each  personality 
s 

trait.  The  following  general  hypothesis  was  tested  for 
each  personality  trait: 

E No  relationship  exists  between  personality 
traits  and  demonstrated  troubleshooting 
ability  of  technicians  with  more  than  six 
months  experience  when  troubleshooting  at 
the  intermediate  maintenance  level. 

E^:  A relationship  exists  between  personality 

traits  and  demonstrated  troubleshooting 
ability  of  technicians  with  more  than  six 
months  experience  when  troubleshooting  at 
the  intermediate  maintenance  level. 

Since  there  was  no  basis  for  predicting  whether  any 
existing  relationship  would  be  direct  or  indirect,  the 
hypothesis  tests  were  conducted  as  two-tailed  tests  at  the 
.05  level  of  significance.  As  indicated  in  Table  3A, 
Responsibility  was  the  only  personality  trait  found  to  be 
significantly  related  to  performance  in  this  subject  group. 
Therefore,  there  is  evidence  that  for  subjects  with  more 
than  six  months  experience,  troubleshooting  at  the  inter- 
mediate maintenance  level,  a relationship  exists  between 
performance  and  personality  test  scores  for  Responsibility. 

of 

For  this  analysis  the  subjects  were  placed  into 
groups  according  to  the  type  of  aid  used  so  that  the  three 
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Analysis  of  the  Correlation 
Personality  Traits  with  Per- 
formance  by  Type  of  Aid 


TABLE  3A 


RESULTS  OF  TESTS  FOR  RELATIONSHIP  BETWEEN  PERSONALITY 
TRAITS  AND  PERFORMANCE  OF  MORE -THAN- SIX-MONTHS- 
EXPERIENCE  SUBJECT  GROUP— INTERMEDIATE 
MAINTENANCE  LEVEL 


Personality  Trait 

Computed 

V 

s 

Computed 

t 

P 

Cautiousness 

. 016 

.062 

.90 

< 

P 

Ascendancy 

. 214 

. 878 

.30 

< 

P 

< .40 

Vigor 

.161 

.625 

.50 

< 

P 

< .60 

Personal  Relations 

-.294 

-1.229 

.20 

< 

P 

< .30 

Sociability 

-.002 

- .008 

. 90 

< 

P 

< 

Original  Thinking 

.098 

.392 

P 

= .70 

Emotional  Stability 

.146 

.589 

.50 

< 

P 

< .60 

Responsibility 

.516 

2. 407* 

.02 

< 

P 

< .05 

Critical  t 

value  = + 2. 

120,  a = .05/ 

df  = 

16. 

*Significant  at  a = .05. 
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subject  groups  now  consisted  of  54  subjects  using  the  FPTA, 
54  subjects  using  the  LTTA,  and  36  subjects  using  the  T.O. 
The  Spearman  rank  correlation  coefficient  test  was  used  to 
determine  if  any  relationships  existed  between  the  eight 
personality  test  scores  and  performance  with  each  type  of 
aid.  The  personality  test  scores  for  all  subjects  are  con- 
tained in  Appendix  C,  Table  C-2.  The  performance  indices 
for  each  type  of  aid  were  taken  from  Appendix  A,  Tables 
A-l,  A-2,  and  A-3.  The  following  is  a discussion  of  the 
analysis  by  type  of  aid. 

Correlation  of  Personality  Traits  and  Performance 
Using  the  FPTA.  The  rank  ordering  of  the  FPTA  performance 
indices  for  all  54  subjects  and  the  rank  ordering  of  the 
eight  personality  test  scores  are  contained  in  Appendix  C, 
Table  C-5.  The  computations  of  the  Separman  and  t sta- 
tistics for  the  FPTA  are  in  Appendix  C,  Table  C-6.  The 
following  general  hypothesis  was  tested  for  each  personality 
trait: 

# .*  No  relationship  exists  between  personality 
traits  and  demonstrated  troubleshooting 
ability  of  technicians  us^ng  FPTA  documenta- 
tion. 

ff  : A relationship  exists  between  personality 

traits  and  demonstrated  troubleshooting 
ability  of  technicians  using  FPTA  documenta- 
tion. 

These  hypothesis  tests  were  conducted  as  two-tailed 
tests  at  the  .05  level  of  significance.  Table  3B  shows 
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TABLE  3B 


RELATIONSHIP  BETWEEN  PERSONALITY  TRAITS  AND  PERFORMANCE 


USING  FPTA: 

RESULTS  OF 

SPEARMAN  TEST 

Personality  Trait 

Computed 

■p 

Computed 

P 

s 

Cautiousness 

.027 

.195 

• 

00 

o 

< 

P 

< .90 

Ascendancy 

.041 

.103 

.90 

< 

P 

Vigor 

.051 

.366 

.70 

< 

P 

< .80 

Personal  Relations 

-.064 

-.461 

.60 

< 

P 

< .70 

Sociability 

-.109 

-.786 

.40  < 

P < 

. 50 

Original  Thinking 

-.063 

-.448 

.60  < 

P < 

. 70 

Emotional  Stability 

-.029 

-.205 

.80  < 

P < 

.90 

Responsibility 

-.012 

-.083 

.90  < 

P 

Critical  t Value  = + 

2*2.0/  ct  = *05/ 

df  = 51 

that  the  null  hypothesis,  R , could  not  be  rejected  for  any 
of  the  personality  traits.  Therefore  it  could  not  be  con- 
cluded that  any  of  the  personality  traits  measured  were 
related  to  the  subjects'  troubleshooting  performance  using 
Fully  Proceduralized  Troubleshooting  Aids. 

Correlation  of  Personality  Trait  and  Performance 
Using  the  LTTA.  The  rank  ordering  of  the  LTTA  performance 
indices  for  all  subjects  is  also  contained  in  Appendix  C, 
Table  C-5.  Appendix  C,  Table  C-7  contains  the  computa- 
tions of  the  Spearman  and  t statistics  for  the  LTTA. 

The  following  general  hypothesis  was  tested  for  each  per- 
sonality trait: 

H Q:  No  relationship  exists  between  personality 

traits  and  demonstrated  troubleshooting 
ability  of  technicians  using  LTTA  documenta- 
tion. 

H- : A relationship  exists  between  personality 

traits  and  demonstrated  troubleshooting 
ability  of  technicians  using  LTTA  documenta- 
tion. 

These  hypothesis  tests  were  conducted  as  two-tailed 
tests  at  the  .05  level  of  significance.  The  results  of 
the  tests,  shown  in  Table  3C,  indicate  that  the  null 
hypothesis,  R Q,  could  be  rejected  for  two  personality 
traits:  Cautiousness  and  Emotional  Stability.  Therefore, 

there  is  evidence  that  Cautiousness  and  Emotional  Stability 
are  related  to  troubleshooting  performance  of  technicians 
using  Logic  Tree  Troubleshooting  Aids. 
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TABLE  3C 


RELATIONSHIP  BETWEEN  PERSONALITY  TRAITS  AND  PERFORMANCE 


USING  LTTA: 

RESULTS 

OF  SPEARMAN 

TEST 

Personality  Trait 

Computed 

r 

s 

Computed 

t 

P 

Cautiousness 

.348 

2.654* 

.01 

< 

P 

< 

.02 

Ascendancy 

.131 

.947 

.30 

< 

P 

< 

.40 

Vigor 

.171 

1.236 

.20 

< 

P 

< 

.30 

Personal  Relations 

.105 

.751 

.40 

< 

P 

< 

.50 

Sociability 

-.137 

-.986 

.30 

< 

P 

< 

.40 

Original  Thinking 

.190 

1.382 

.10 

< 

P 

< 

.20 

Emotional  Stability 

. 331 

2.506* 

.01 

< 

P 

< 

.02 

Responsibility 

.211 

1.  537 

.10 

< 

P 

< 

.20 

Critical  t value  = + 2. 

01,  a = .05, 

df  = : 

51 

♦Significant  at  a = .05. 
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Correlation  of  Personality  Traits  and  Performance 
Using  the  T.O.  The  rank  ordering  of  the  performance 
indices  for  the  T.O.  and  the  rank  ordering  of  the  sub- 
jects' eight  personality  test  scores  are  contained  in 
Appendix  C,  Table  C-8.  Since  the  zero-experience  subjects 
(Keesler  graduates)  were  not  evaluated  using  the  T.O.s, 
this  subject  group  contained  only  36  subjects.  The  Spear- 
man and  t computations  for  the  T.O.  are  contained  in 
Appendix  C,  Table  C-9.  The  following  general  hypothesis 
was  tested  for  each  personality  trait: 

tf  : No  relationship  exists  between  personality 

traits  and  demonstrated  troubleshooting 
ability  of  technicians  using  T.O.  documenta- 
tion. 

H-:  A relationship  exists  between  personality 

traits  and  demonstrated  troubleshooting 
ability  of  technicians  using  T.O.  documenta- 
tion. 

Again,  the  hypothesis  tests  were  conducted  as  two- 
tailed  tests  at  the  .05  level  of  significance.  As  indi- 
cated by  the  test  results  in  Table  3D,  the  null  hypothesis 
could  not  be  rejected  for  any  of  the  personality  traits  at 
the  .05  level  of  significance.  Therefore,  it  could  not  be 
concluded  that  any  of  the  eight  personality  traits  were 
related  to  troubleshooting  ability  of  technicians  using 
Technical  Orders. 
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TABLE  3D 

RELATIONSHIP  BETWEEN  PERSONALITY  TRAITS  AND  PERFORMANCE 


USING  T.O. : 

: RESULTS 

OF  SPEARMAN 

TEST 

Personality  Trait 

Computed 

V 

s 

Computed 

t 

P 

Cautiousness 

.286 

1.742 

in 

o 

• 

< 

P 

< 

.10 

Ascendancy 

-.139 

-.819 

. 40 

< 

P 

< 

.50 

Vigor 

.064 

.374 

.70 

< 

P 

< 

.80 

Personal  Relations 

-.293 

-1.888 

.05 

< 

P 

< 

.10 

Sociability 

-.268 

-1.625 

.10 

< 

P 

< 

.20 

Original  Thinking 

-.042 

- .244 

. 80 

< 

P 

< 

. 90 

Emotional  Stability 

.165 

.976 

.30 

< 

P 

< 

.40 

Responsibility 

. 324 

1.  990 

.05 

< 

P 

< 

.10 

Critical  t Value  = + 2. 

034,  a = .05,  df  = 

34. 
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Relationship  Between  Aptitude  Scores  and 
Troubleshooting  Performance 

The  fourth  research  proposition  was  that  "There 
is  a positive  relationship  between  AQE  aptitude  scores  and 
demonstrated  troubleshooting  ability. " The  objective  of 
this  proposition  was  to  determine  whether  a technician's 
AQE  scores  are  a good  predictor  of  how  well  he  will  per- 
form on  the  job. 

Since  the  propositions  regarding  native  ability 
and  differences  in  performance  between  different  experience- 
level  groups  were  not  supported,  the  tests  for  relation- 
ships between  aptitude  and  performance  were  conducted  in 
the  same  manner  as  the  personality  correlation  tests, 
rather  than  as  originally  intended.  That  is,  the  AQE 
scores  were  first  correlated  with  the  average  performance 
indices  of  the  more-than-six-months-experience  group  of 
the  intermediate  maintenance  level,  and  secondly,  AQE 
scores  were  correlated  with  all  the  subjects'  performance 
indices  with  each  type  of  troubleshooting  aid. 

Analysis  of  the  More-Than-Six- 
Months- Experience-Level  Group  at 
the  Intermediate  Maintenance  Level 

The  performance  indices  used  in  this  analysis  were 
computed  as  an  average  performance  index  across  all  three 
types  of  aids  used.  The  average  performance  indices  were 
the  same  as  those  used  in  testing  the  third  research 
proposition  and  are  shown  in  Appendix  C,  Table  C-l.  The 
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Spearman  rank  correlation  coefficient  test  was  used  to  test 
for  relationships  between  the  average  performance  indices 
and  the  subjects'  scores  in  each  of  the  four  aptitude 
clusters:  General,  Administrative,  Mechanical,  and  Elec- 
tronics. The  AQE  scores  for  all  subjects  are  shown  in 
Appendix  D,  Table  D-l.  The  rank  ordering  of  the  average 
performance  indices  and  the  AQE  scores  are  contained  in 
Appendix  D,  Table  D-2.  Appendix  D,  Table  D-3,  contains 
the  computations  of  the  Spearman  and  t statistics  for 
each  aptitude  cluster.  The  following  general  hypothesis 
was  tested  for  each  aptitude  cluster: 

H : No  positive  relationship  exists  between  AQE 

scores  and  demonstrated  troubleshooting 
ability  of  technicians  with  more  than  six 
months  experience  at  the  intermediate  main- 
tenance level. 

S : A positive  relationship  exists  between  AQE 

scores  and  demonstrated  troubleshooting 
ability  of  technicians  with  more  than  six 
months  experience  at  the  intermediate  main- 
tenance level. 

Since  the  relationship  was  predicted  as  positive, 
these  tests  were  conducted  as  one-tailed  tests  at  the  . 05 
level  of  significance.  As  indicated  in  Table  4A,  the  null 
hypothesis,  , was  not  rejected  for  any  of  the  aptitude 
areas.  Although  the  trend  was  in  the  right  direction  for 
electronics  aptitude,  it  could  not  be  concluded,  from  the 
statistical  test  and  the  data,  that  any  of  the  AQE  apti- 
tude scores  were  positively  related  to  intermediate 
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TABLE  4A 


* 


RELATIONSHIP  BETWEEN  APTITUDE  AND  PERFORMANCE: 
MORE- THAN- S I X-MONTHS - EXP  ERI ENC  E- LEVEL  GROUP 
ON  INTERMEDIATE  MAINTENANCE 
LEVEL  PROBLEMS 


Computed  Computed 

Aptitude  Cluster  v t p 


General 

-.178 

- .701 

Administrative 

-.333 

-1.368 

Mechanical 

-.196 

- .773 

Electronics 

.091 

.355  .70  < p < .80 

Critical 

t Value  = 1. 

753,  a = .05,  df  = 15. 

aSince  the  null  hypothesis  would  never  be  rejected 
with  a negative  t value  in  a one-tailed  test  for  positive 
relationship,  all  p values  are  not  shown. 

maintenance  level  troubleshooting  ability  of  technicians 
with  more  than  six  months  experience. 

Analysis  of  the  Correlation 
of  Aptitude  Scores  with 
Performance  by  Type  of 
Troubleshooting  Aid 

For  this  analysis,  the  subjects  were  again  grouped 
by  type  of  troubleshooting  aid  so  that  the  three  subject 
groups  consisted  of  54  subjects  using  the  FPTA,  54  sub- 
jects using  the  LTTA,  and  36  subjects  using  the  T.O.  The 
Spearman  rank  correlation  test  was  used  to  determine  if 
any  positive  relationships  existed  between  the  four  aptitude 
scores  and  performance  with  each  type  of  aid.  The  AQE 
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scores  for  all  subjects  are  contained  in  Appendix  D,  Table 
D-l.  The  performance  indices  for  each  type  of  aid  were 
taken  from  Appendix  A,  Tables  A-l,  A-2,  and  A-3.  The 
following  is  a discussion  of  the  analysis  by  type  of  aid. 

Correlation  of  Aptitude  Scores  and  Performance 
Using  the  FPTA.  The  rank  ordering  of  the  FPTA  performance 
indices  for  all  54  subjects  and  the  rank  ordering  of  the 
four  aptitude  test  scores  are  contained  in  Appendix  D, 

Table  D-4.  The  computations  of  the  Spearman  and  t sta- 
tistics are  contained  in  Appendix  D,  Table  D-5.  The 
following  general  hypothesis  was  tested  for  each  aptitude 
cluster: 

3 : No  positive  relationship  exists  between  AQE 

u scores  and  demonstrated  troubleshooting  ability 
of  technicians  using  FPTA  documentation. 

3 : A positive  relationship  exists  between  AQe 

scores  and  demonstrated  troubleshooting  ability 
of  technicians  using  FPTA  documentation. 

These  hypothesis  tests  were  conducted  as  one-tailed 
tests  at  the  .05  level  of  significance.  Table  4B  not  only 
shows  that  the  null  hypothesis,  H could  not  be  rejected 
for  any  of  the  aptitude  clusters,  but  also  that  the  trend 
was  in  the  opposite  direction  from  the  predicted  relation- 
ship (i.e.,  a slight,  nonsignificant,  negative  relation- 
ship was  found).  Therefore,  it  could  not  be  concluded, 
at  the  .05  level  of  significance,  that  any  of  the  aptitude 
scores  were  positively  related  to  the  troubleshooting  ability 

104 


TABLE  4B 


RELATIONSHIP  BETWEEN  APTITUDE  AND 


PERFORMANCE 

USING  FPTA 

Aptitude  Cluster 

Computed 

V 

s 

Computed 

t 

a 

P 

General 

-.020 

- .145 

Administrative 

-.172 

-1.247 

Mechanical 

-.032 

- .228 

Electronic 

-.217 

-1.589 

Critical  t Value  = 1.678,  a = .05,  df  = 51. 

aSince  the  null  hypothesis  would  never  be  rejected 
with  a negative  t value  in  a one-tailed  test  for  positive 
relationship,  p values  are  not  shown. 

of  technicians  using  Fully  Proceduralized  Troubleshooting 
Aids. 

Correlation  of  Aptitude  Scores  and  Performance 
Using  the  LTTA.  The  rank  ordering  for  the  LTTA  performance 
indices  is  also  contained  in  Appendix  D,  Table  D-4.  The 
computations  of  the  Spearman  r and  t statistics  for  the 
LTTA  are  shown  in  Appendix  D,  Table  D-6.  The  following 
general  hypothesis  was  tested  for  each  aptitude  cluster: 

g : No  positive  relationship  exists  between  AQE 

0 scores  and  demonstrated  troubleshooting 

ability  of  technicians  using  LTTA  documenta- 
tion. 
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E •.  A positive  relationship  exists  between  AQE 
scores  and  demonstrated  troubleshooting 
ability  of  technicians  using  LTTA  documenta- 
tion. 

These  hypothesis  tests  were  conducted  as  one-tailed 
tests  at  the  .05  level  of  signif xcance.  Table  4C  indicates 
that  the  null  hypothesis  was  rejected  for  only  one  aptitude 
cluster — Administrative.  Therefore,  the  data  and  sta- 
tistical tests  provide  evidence  that  the  Administrative 
aptitude  test  score  is  positively  related  to  a technician's 
performance  using  the  LTTA.  For  the  other  three  AQE 
aptitude  clusters,  relationships,  although  slight  and  non- 
significant, are  in  the  opposite  direction  from  that  pre- 
dicted. 


TABLE  4C 

RELATIONSHIP  BETWEEN  APTITUDE  AND  PERFORMANCE 
USING  THE  LTTA 


Aptitude  Cluster 

Computed 

r 

s 

Computed 

t 

a 

P 

General 

-.036 

- .254 

Administrative 

.277 

2.056* 

.02  < p < .05 

Mechanical 

-.002 

- .013 

Electronics 

-.003 

- .019 

Critical 

t Value  = 1.678, 

a — -05,  df 

= 51. 

♦Significant  at  a * .05. 

aSince  the  null  hypothesis  would  never  be  rejected 
with  a negative  t value  in  a one-tailed  test  for  positive 
relationship,  all  p values  are  not  shown. 
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Correlation  of  Aptitude  Scores  and  Performance 


Using  the  T.O.  The  rank  ordering  of  the  performance 
indices  for  the  T.O.  and  the  rank  ordering  of  the  sub- 
jects' four  aptitude  test  scores  are  contained  in  Appendix 
D,  Table  D-7.  Since  the  zero-experience  subjects  were  not 
evaluated  using  the  T.O.,  this  subject  group  consists  of 
only  36  subjects.  Appendix  D,  Table  D-8,  contains  the  com- 
putations of  the  Spearman  and  t for  the  T.O.  The  follow- 
ing general  hypothesis  was  tested  for  each  aptitude  cluster: 

••  No  positive  relationship  exists  between  AQE 
scores  and  demonstrated  troubleshooting 
ability  of  technicians  using  T.O.  documenta- 
tion. 

B.'.  A positive  relationship  exists  between  AQE 
scores  and  demonstrated  troubleshooting 
ability  of  technicians  using  T.O.  documenta- 
tion. 

The  hypothesis  tests  were  conducted  as  one-tailed 
tests  at  the  .05  level  of  significance.  The  results  of 
the  tests,  shown  in  Table  4D,  indicate  that,  although  the 
trend  was  in  the  predicted  direction,  the  null  hypothesis, 

H could  not  be  rejected  for  any  of  the  four  aptitude 
clusters.  Therefore,  it  could  not  be  concluded  that  any 
of  the  AQE  aptitude  test  scores  were  positively  related  to 
troubleshooting  performance  of  technicians  using  T.O. 
documentation . 

Limitations  of  Analysis 

The  analysis  of  relationships  between  the  AQE 
scores  and  troubleshooting  performance  must  be  viewed  with 
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TABLE  4D 


RELATIONSHIP  BETWEEN  APTITUDE  AND  PERFORMANCE 
USING  THE  T.O. 


Aptitude  Cluster 

Computed 

r 

s 

Computed 

t 

P 

General 

. 044 

.252 

.80  < p 

< .90 

Administrative 

.062 

.355 

.70  < p 

< .80 

Mechanical 

.118 

.681 

.50  < p 

< .60 

Electronics 

.2410 

1.425 

. 10  < p 

< .20 

Critical  t 

Value  = 1.693 

, a = .05,  df 

= 34. 

respect  to  a major 

limitation. 

The  subjects  which  were 

sampled  for  the  performance  data  had  been  placed  in  this 
career  field  based  on  AQE  scores.  In  particular,  a minimum 
score  of  80  was  required  on  the  Electronics  aptitude 
cluster.  Therefore,  the  subjects'  had  already  been  through 
a "screening"  process  based  on  aptitude  scores.  Due  to 
this  prior  screening,  the  subjects'  AQE  scores  are  not 
representative  of  the  Air  Force  population  as  a whole,  par- 
ticularly with  respect  to  Electronics  aptitude.  The  result 
of  the  Spearman  tests  on  relationship  between  aptitude  and 
performance  must  be  viewed  with  this  limitation  in  mind. 
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CHAPTER  V 


SUMMARY  OF  FINDINGS,  CONCLUSIONS,  AND 
RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

This  chapter  summarizes  the  findings  of  this 
research  effort,  presents  the  conclusions,  and  offers  some 
general  recommendations  for  further  research.  The  research 
findings  are  presented  within  the  context  of  the  three  pri- 
mary research  objectives  as  presented  in  Chapter  I. 

Research  Findings 
First  Research  Objective 

The  first  objective  of  this  research  was  to  con- 
firm that  there  exist  individuals  who  possess  native 
abilities  for  troubleshooting  tasks. 

The  initial  attempt  at  accomplishing  this  objective 
was  unsuccessful.  It  could  not  be  shown  through  statistical 
tests  of  the  data  that  the  relative  performance  of  the  sub- 
jects within  each  experience- level  group  was  consistent 
using  various  types  of  troubleshooting  aids. 

In  the  additional  analysis,  it  was  found  that  the 
type  of  troubleshooting  aid  had  a significant  effect  on 
performance  in  the  zero-experience  group  and  the  six- 
months-or-less-experience  group  but  not  in  the  more-than- 
six-months-experience  group.  And,  when  the  standard  times 
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used  for  replacement  of  spare  parts  was  excluded  from  the 
performance  indices,  the  more-than-six-months-experience 
group  demonstrated  consistency  of  performance  using  vari- 
ous types  of  aids.  Additionally,  when  the  subjects'  per- 
formance was  analyzed  at  the  two  different  maintenance 
levels,  it  was  again  the  more- than- six-months-experience 
group  which  demonstrated  consistency  of  performance  using 
various  types  of  aids  at  the  intermediate  maintenance 
level. 

Second  Research  Objective 

The  second  research  objective  was  to  determine 
whether  personality  assessment  techniques  could  be  used  to 
identify  potential  troubleshooting  capability  prior  to 
actual  training  and  career  field  assignment. 

Originally,  it  was  intended  to  accomplish  this 
objective  by  identifying  those  personality  traits  which 
were  significantly  related  to  the  performance  of  the  sub- 
jects in  each  group.  This  strategy  was  based  on  the  belief 
that  the  performance  of  the  groups  would  be  significantly 
different  and  that  the  subjects  would  demonstrate  con- 
sistency of  relative  performance  using  various  types  of 
aids. 

Although  these  beliefs  were  not  supported,  some 
insight  was  gained  through  additional  analysis.  First, 
since  the  more-than-six-months-experience  group  did 
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demonstrate  consistency  of  relative  performance  using 
various  types  of  aids  at  the  intermediate  maintenance 
level,  a correlation  of  the  performance  and  the  personality 
test  scores  of  this  group  was  conducted.  Responsibility 
was  the  only  personality  trait  found  to  be  significantly 
related  to  performance  in  this  group. 

Secondly,  since  no  significant  differences  were 
found  among  the  performance  of  the  different  experience- 
level  subject  groups,  all  subjects  were  placed  into  one 
sample  group  and  their  performance  by  type  of  trouble- 
shooting aid  was  correlated  with  their  personality  test 
scores . 

No  personality  traits  were  found  to  be  signifi- 
cantly related  to  performance  of  the  subjects  using  the 
FPTA  or  the  T.O.  The  performance  of  subjects  using  the 
LTTA  was  found  to  be  significantly  related  to  Cautiousness 
and  Emotional  Stability. 

Third  Research  Objective 

The  third  and  final  objective  of  this  research  was 
to  confirm  the  belief  that  aptitude  measures  are  an  accu- 
rate indicator  of  actual  troubleshooting  performance. 

When  the  performance  of  the  more-than-six-months- 
experience  group  was  correlated  with  the  AQE  scores  of 
that  group,  it  was  found  that  none  of  the  aptitude  test 
scores  were  significantly  related  to  performance.  However, 
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it  was  interesting  to  note  that  although  none  of  the  rela- 
tionships were  significant,  the  Electronics  aptitude 
cluster  was  the  only  one  to  indicate  a positive  correla- 
tion. 

The  correlation  of  aptitude  scores  with  performance 
by  type  of  troubleshooting  aid  demonstrated  that  none  of 
the  AQE  test  scores  were  significantly  related  to  perform- 
ance of  subjects  using  the  FPTA  or  the  T.O.  A significant 
relationship,  however,  was  found  between  the  Administrative 
aptitude  cluster  score  and  performance  of  subjects  using 
the  LTTA. 


Research  Conclusions 

The  following  conclusions  were  drawn  from  the 
research  findings: 

1.  Overall,  there  was  little  evidence  that  indi- 
viduals who  perform  well  using  one  type  of  troubleshooting 
aid,  tend  to  perform  well  using  other  types  of  trouble- 
shooting aids.  Thus,  it  could  not  be  concluded  that  some 
individuals  possess  higher  levels  of  native  ability  for 
troubleshooting  than  do  others.  The  additional  analysis 
which  was  conducted  in  an  effort  to  determine  if  native 
ability  was  actually  nonexistent,  or  if  it  had  merely 
been  masked  by  other  factors,  revealed  that  the  more-than- 
six-months-experience  group  tended  to  perform  consistently 
using  the  different  types  of  troubleshooting  aids. 


Therefore,  it  is  concluded  that  experience  is  an  important 
factor  in  consistency  of  performance  using  various  types 
of  aids.  Thus,  it  well  may  be  that  some  individuals 
actually  possess  higher  levels  of  native  ability  for 
troubleshooting  than  others,  but  that  these  differences  do 
not  become  evident  until  the  individuals  have  acquired 
several  months  of  troubleshooting  experience. 

2.  Although  the  trend  was  in  the  predicted  direc- 
tion (i.e.,  performance  improved  as  experience  increased), 
differences  between  the  average  performances  of  the  three 
different  experience- level  groups  were  not  significant. 
Therefore,  it  is  concluded  that  the  use  of  more  procedural- 
ized  aids,  such  as  the  FPTA  and  LTTA,  allows  technicians 
with  little  or  no  experience  to  troubleshoot  almost  as 
effectively  as  experienced  technicians.  With  this  being 
the  case,  then  the  Air  Force  should  concentrate  on  develop- 
ing proceduralized  aids  which  would  enable  technicians  with 
less  training  and  experience  to  become  productive. 

3.  There  was  no  evidence  that  any  of  the  personal- 
ity traits  measured  by  the  Gordon  instruments  were  con- 
sistently related  to  troubleshooting  performance.  There- 
fore, the  conclusion  is  that  personality  testing  may  not 

be  suitable  for  selecting  personnel  for  entry  into  this 
particular  career  field.  However,  since  differences  in 
native  ability  may  not  be  discernable  until  after  indi- 
viduals have  acquired  a certain  level  of  experience,  the 


usefulness  of  personality  testing  as  a selection  device 
can  not  be  absolutely  precluded. 

4.  No  unqualified  conclusions  can  be  drawn  from 
the  research  findings  concerning  the  relationship  between 
aptitude  as  measured  by  the  AQE  and  troubleshooting  per- 
formance. The  subjects  had  already  been  screened  based 
on  AQE  scores  and,  therefore,  were  not  considered  to  be 
a representative  sample. 

Recommendations  for  Future  Research 

One  area  of  interest  for  future  research  is  addi- 
tional testing  for  consistency  of  performance  of  experi- 
enced technicians  using  various  types  of  aids.  If  it  can 
be  shown  that  experienced  technicians  do  indeed  perform 
consistently  regardless  of  the  type  of  aid  used,  then 
additional  analysis  can  be  conducted  to  determine  if  the 
personality  trait.  Responsibility,  is  actually  related  to 
performance  of  these  technicians.  If  so,  then  the  presence 
of  the  personality  trait  can  be  used  as  a screening  factor 
in  selecting  of  personnel  for  entry  into  the  career  field. 

A second  recommendation  is  that  additional  testing 
be  conducted  to  determine  exactly  how  little  formal  train- 
ing is  required  to  enable  technicians  to  troubleshoot 
effectively  using  fully  proceduralized  aids.  This  deter- 
mination must  be  made  in  order  to  adapt  current  training 
programs  to  the  use  of  fully  proceduralized  aids.  It  would 


114 


also  enable  the  Air  Force  to  better  evaluate  the  potential 
benefits  of  using  fully  procedural! zed  aids. 


Since  the  samples  were  considered  to  be  nonrepre- 
sentative with  respect  to  AQE  scores,  no  unqualified  con- 
clusions could  be  made  concerning  the  relationship  between 
aptitude  test  scores  and  troubleshooting  performance. 
Therefore,  it  is  recommended  that  additional  research  be 
conducted  in  this  area  using  sample  groups  which  have  not 
been  screened  by  the  selection  procedures  based  on  AQE 
scores.  If  it  can  be  shown  that  aptitude  scores,  par- 
ticularly in  the  Electronics  aptitude  cluster,  definitely 
are  not  related  to  troubleshooting  performance  with  fully 
proceduralized  aids,  then  the  Air  Force  would  have  avail- 
able a larger  number  of  personnel  who  could  be  productive 
as  electronics  technicians. 
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APPENDICES 


Subjects  throughout  the  appendices  are  numbered  in 
this  manner:  subjects  1 through  18 — Zero  Experi- 

ence; subjects  19  through  36 — Less-Than-Six-Months 
Experience;  subjects  37  through  54 — More-Than-Six- 
Months  Experience. 


*1 


APPENDIX  A 

COMPUTATIONS  OF  PERFORMANCE  INDICES 
AND  FRIEDMAN  STATISTICS 
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TABLE  A- 7 


PERFORMANCE 

INDICES  RANKINGS 
ZERO- EXPERIENCE 

FOR  FPTA  AND  LTTA: 
GROUP 

Subject 

Number 

FPTA 

LTTA 

1 

1 

2 

2 

1 

2 

3 

2 

1 

4 

1 

2 

5 

1 

2 

6 

2 

1 

7 

1 

2 

8 

1 

2 

9 

2 

1 

10 

1 

2 

11 

1 

2 

12 

1 

2 

13 

1 

2 

14 

2 

1 

15 

1 

2 

16 

1 

2 

17 

1 

2 

18 

1 

2 

Total 

22 

32 

xr  = (18)^2)  (3)  (1'508)  “ (3)  (18)  (3)  = 5.555 
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TABLE  A- 8 


PERFORMANCE  INDICES  RANKINGS  FOR  FPTA,  LTTA,  AND  T.O.: 
SIX- MONTHS-OR- LESS- EXPERIENCE  GROUP 


Subject 

Number 

FPTA 

LTTA 

T.O. 

19 

2 

1 

3 

20 

1 

2 

3 

21 

2 

1 

3 

22 

2 

1 

3 

23 

2 

1 

3 

24 

2 

1 

3 

25 

2 

1 

3 

26 

1 

2 

3 

27 

1 

2 

3 

28 

1 

2 

3 

29 

1 

3 

2 

30 

1 

2 

3 

21 

3 

1 

2 

32 

1 

2 

3 

33 

1 

2 

3 

34 

1 

2 

3 

35 

2 

1 

3 

36 

2 

1 

3 

Total 

28 

28 

52 

Xr  = fiSHS)  (4)  (4/  272)  - (3)  (18)  (4)  = 21.333 
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TABLE  A- 9 


PERFORMANCE  INDICES  RANKINGS  FOR  FPTA , LTTA,  AND  T.O.: 
MORE- THAN- SIX- MONTHS- EXPER I ENCE  GROUP 


Subject 

Number 

FPTA 

LTTA 

T.O. 

37 

1 

2 

3 

38 

3 

1 

2 

39 

3 

2 

1 

40 

3 

2 

1 

41 

1 

2 

3 

42 

2 

3 

1 

43 

1 

2 

3 

44 

2 

3 

1 

45 

3 

1 

2 

46 

2 

3 

1 

47 

2 

1 

3 

48 

2 

1 

3 

49 

2 

1 

3 

50 

2 

1 

3 

51 

2 

3 

1 

52 

2 

1 

3 

53 

1 

3 

2 

54 

2 

3 

1 

Total 

36 

35 

37 

2 

xr 


12 

(18)  (3)  (4) 


(3890) 


(3)  (18)  (4)  = .111 
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TABLE  A- 16 
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139 


Number  of  problems  solved  correctly. 


PERFORMANCE  INDICES:  SIX-MONTHS-OR-LESS-EXPERIENCE  GROUP- 
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APPENDIX  B 

DIFFERENCE  IN  PERFORMANCE  BETWEEN  DIFFERENT 
EXPERIENCE-LEVEL  GROUPS 
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TABIE  B-l 


AVERAGE  PERFORMANCE  INDICES:  ZERO- EXPERIENCE  GROUP 


FPTAa 

LTTAb 

APIC 

Sub j ect 
Number 

PI 

PI 

API 

1 

39.167 

101.633 

70.400 

2 

51.676 

90.344 

71.010 

3 

40.817 

34.272 

37.545 

4 

29.905 

44.150 

37.028 

5 

58.170 

91.192 

74.681 

6 

54.453 

49.197 

51.825 

7 

27.645 

44.389 

36.017 

8 

27.724 

68.000 

47.862 

9 

42.400 

40.325 

41.363 

10 

55.393 

69.437 

62.665 

11 

24.457 

66.597 

45.527 

12 

29.395 

36.944 

33.170 

13 

32.098 

40.503 

36.301 

14 

57 . 447 

52.637 

55.042 

15 

38.226 

101.775 

70.001 

16 

33.531 

88.371 

60.951 

17 

32.379 

71.930 

52.155 

18 

45.447 

55.123 

50.285 

^Performance  index  with  spare  parts  time  for  FPTA. 
^Performance  index  with  spare  parts  time  for  LTTA. 


c 


Average  Performance  Index  = 


FPTA  + LTTA 
2 
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TABLE  B-2 

AVERAGE  PERFORMANCE  INDICES: 

SI X-MONTHS-OR- LESS- EXPERIENCE  GROUP 


FPTA 

LTTA 

T.O. 

r--  ■ 1 — 

API 

Subject 

a 

b 

c 

d 

Number 

pi 

PI 

PI 

API 

19 

54.500 

42.600 

80.000 

59.033 

20 

23.367 

49.500 

129.000 

67.289 

21 

25.500 

13.800 

24.700 

23.000 

22 

45.260 

35.300 

62.667 

47.742 

23 

28.537 

5.900 

25. 570 

20.002 

24 

32.000 

20.900 

35.200 

29.367 

25 

33.200 

25.000 

24.400 

27.533 

26 

23.337 

65.875 

121.056 

69.756 

27 

27.400 

36.250 

128.750 

64.133 

28 

31.400 

33.750 

64.250 

43.133 

29 

23.334 

87.667 

62.242 

57.750 

30 

32.400 

38.400 

55.333 

42. 044 

31 

36. 600 

22.800 

24.400 

27.933 

32 

22.000 

20.000 

154.000 

65.333 

33 

20.200 

33.000 

181.500 

78.233 

34 

33.800 

32.800 

112.689 

59.763 

35 

31.750 

35.250 

62.356 

43.119 

36 

53.054 

22.663 

48.221 

41.313 

Performance  index  with  spare  parts  time  for  FPTA. 
^Performance  Index  with  spare  parts  time  for  LTTA. 
"Performance  index  with  spare  parts  time  for  T.O. 


Average  Performance  Index  = 


FPTA  + LTTA  + T.O. 
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TABLE  B-3 


AVERAGE  PERFORMANCE  INDICES: 
MORE- THAN- SIX- MONTHS- EXP ERIENCE  GROUP 


Subject 

Number 

FPTA 

LTTA 

T.O. 

API 

Pia 

Plb 

■ 

PIC 

APId 

37 

22.600 

66.004 

123.725 

70.776 

38 

46.000 

20. 600 

30.800 

32.467 

39 

46.660 

32.423 

22.043 

33.709 

40 

40.923 

27.617 

21.400 

29.980 

41 

36.170 

40.900 

75.960 

51.010 

42 

43.367 

60.000 

41.000 

48.122 

43 

89. 522 

104.000 

158.150 

117.224 

44 

18.600 

45.492 

8.200 

24.131 

45 

48.490 

25.803 

27.800 

34.031 

46 

35.660 

37.413 

30.553 

34.542 

47 

39.763 

32.854 

61.958 

44.858 

48 

29.530 

25.770 

79.972 

45.091 

49 

50.296 

47.880 

65.404 

54.527 

50 

33.400 

20.327 

35.157 

29.628 

51 

48.563 

66.200 

44.250 

53.004 

52 

29.083 

17.083 

84.994 

43.720 

53 

16.200 

27.400 

18.000 

20.533 

54 

23.127 

35.607 

22.400 

19.578 

Performance  index  with  spare  parts  time  for  FPTA. 

Performance  index  with  spare  parts  time  for  LTTA. 

Performance  index  with  spare  parts  time  for  T.O. 

d _ „ , . FPTA  + LTTA  + T.O. 

Average  Performance  Index  = . 
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TABLE  B-4 


PERFORMANCE  RANKINGS  AND  KRUSKAL- WALLIS  COMPUTATIONS 


Zero- Experience 
Group 

Six-Months-or 

Group 

-Less 

More-Than-Six -Months 
Group 

Subject 

Number 

Rank 

Subject 

Number 

Rank 

Subject 

Number 

Rank 

1 

49 

19 

40 

37 

50 

2 

51 

20 

46 

38 

11 

3 

19 

21 

4 

39 

13 

4 

18 

22 

29 

40 

10 

5 

52 

23 

2 

41 

33 

6 

34 

24 

8 

42 

31 

7 

16 

25 

7 

43 

54 

8 

30 

26 

47 

44 

5 

9 

21 

27 

44 

45 

14 

10 

43 

28 

24 

46 

15 

11 

28 

29 

39 

47 

26 

12 

12 

30 

22 

48 

27 

13 

17 

31 

6 

49 

37 

14 

38 

32 

45 

50 

9 

15 

48 

33 

53 

51 

36 

16 

42 

34 

41 

52 

25 

17 

35 

35 

23 

53 

3 

18 

32 

36 

20 

54 

1 

*1  = 

585 

R2 

= 500 

R3  = 400 

H = 

. 12  ( 
(54) (55) 

5852  5002 

18  18 

4002 

18 

■)  - 3(55) 

= 3.850 
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APPENDIX  C 

SPEARMAN  RANK  CORRELATION  COEFFICIENT: 
PERSONALITY  TRAIT  SCORES  WITH 
PERFORMANCE  INDICES 
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TABLE  C-l 


Average  performance  index 


Personality 

Traits 

1 

1 10* 

11 

12 

13 

14 

15 

16 

17 

18 

Cautiousness 

- 

23 

19 

23 

29 

28 

29 

20 

26 

Ascendancy 

- 

23 

27 

24 

20 

19 

10 

20 

24 

Vigor 

- 

30 

31 

22 

21 

36 

15 

20 

24 

Personal 

Relations 

- 

19 

20 

21 

31 

20 

30 

31 

29 

Sociability 

- 

19 

17 

19 

22 

17 

21 

21 

k 

26 

Original 

Thinking 

- 

22 

30 

30 

35 

28 

20 

29 

19 

Emotional 

Stability 

- 

20 

28 

22 

26 

32 

20 

22 

27 

Responsibility 

- 

21 

20 

21 

31 

31 

22 

25 

27 

♦Subject  10  did  not  have  Gordon  Personality  Test  scores. 
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TABLE  C-2 — Continued 


Personality 

Traits 

Subject  Number 

19 

20 

21 

22 

23 

24 

25 

26 

27 

Cautiousness 

22 

22 

33 

27 

30 

25 

30 

18 

20 

Ascendancy 

25 

20 

25 

25 

11 

20 

16 

17 

25 

Vigor 

26 

25 

33 

31 

16 

17 

23 

18 

22 

Personal 

Relations 

26 

25 

19 

28 

27 

35 

19 

19 

34 

Sociability 

19 

18 

16 

24 

10 

20 

9 

18 

32 

Original 

Thinking 

23 

24 

27 

34 

19 

17 

22 

19 

32 

Emotional 

Stability 

29 

25 

27 

26 

26 

30 

27 

18 

20 

Responsibility 

20 

23 

30 

33 

24 

25 

23 

18 

23 

Personality 

Traits 

Subject  Number 

28 

29 

30 

31 

32 

33 

34 

35 

36 

Cautiousness 

29 

30 

31 

32 

22 

23 

23 

29 

34 

Ascendancy 

20 

25 

28 

22 

26 

26 

11 

21 

9 

Vigor 

19 

26 

31 

25 

31 

25 

18 

22 

29 

Personal 

Relations 

26 

18 

24 

25 

26 

26 

14 

23 

20 

Sociability 

20 

24 

22 

20 

23 

20 

8 

18 

14 

Original 

Thinking 

17 

20 

26 

36 

33 

32 

19 

33 

25 

Emotional 

Stability 

27 

27 

30 

32 

34 

32 

25 

33 

22 

Responsibi lity 

25 

25 

28 

29 

25 

30 

20 

32 

27 
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TABLE  02 — Continued 


Personality 

Traits 

Subject  Number 

37 

38 

39 

40 

41 

42 

43 

44 

45 

Cautiousness 

28 

17 

28 

28 

22 

30 

30 

18 

24 

Ascendancy 

8 

15 

13 

24 

14 

10 

13 

7 

27 

Vigor 

25 

27 

21 

24 

26 

17 

12 

13 

32 

Personal 

Relations 

24 

19 

22 

34 

23 

19 

25 

14 

17 

Sociability 

15 

14 

13 

18 

14 

15 

14 

13 

19 

Original 

Thinking 

19 

24 

22 

32 

23 

11 

23 

11 

35 

Emotional 

Stability 

21 

11 

25 

34 

19 

27 

24 

14 

27 

Responsibility 

24 

20 

30 

32 

19 

22 

15 

26 

28 

Personality  l 

Traits 

Subject  Number 

46 

47 

48 

49 

50 

51 

52 

53 

54 

Cautiousness 

33 

31 

37 

17 

34 

22 

33 

29 

34 

Asc  endancy 

23 

19 

19 

31 

20 

29 

25 

24 

19 

Vigor 

19 

19 

22 

27 

18 

33 

22 

31 

29 

Personal 

Relations 

26 

29 

35 

32 

18 

24 

33 

17 

26 

Sociability 

21 

19 

20 

31 

20 

35 

20 

16 

19 

Original 

Thinking 

22 

27 

24 

26 

15 

30 

24 

33 

23 

ESootional 

Stability 

29 

34 

35 

22 

32 

13 

33 

32 

26 

Responsibility 

27 

28 

34 

22 

24 

20 

29 

30 

34 
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TABLE  03 


SPEARMAN  RANK  ORDERING — API  AND  GORDON  TRAITS — 
INTERMEDIATE  LEVEL  MAINTENANCE:  MORE- THAN- SIX- MONTHS  GROUP 


API  and 

Personality 

Traits 

Subject  Number 

37 

38 

39 

40 

41 

42 

43 

44 

45 

a 

API 

17 

6 

8 

5 

15 

16 

18 

2 

9 

Cautiousness 

11 

17.5 

11 

11 

15.4 

7.5 

7.5 

16 

13 

Ascendancy 

17 

12 

14.5 

5.5 

13 

16 

14.5 

18 

3 

Vigor 

8 

5.5 

12 

9 

7 

16 

18 

17 

2 

Personal 

Relations 

9.5 

13.5 

12 

2 

11 

13.5 

8 

18 

16.5 

Sociability 

12.5 

15 

17.5 

10 

15 

12.5 

15 

17.5 

8 

Original 

Thinking 

15 

8 

13.5 

3 

11 

17.5 

11 

17.5 

1 

Emotional 

Stability 

14 

18 

11 

2.5 

15 

8.5 

12 

16 

8.5 

Responsibility 

11.5 

15.5 

4.5 

3 

17 

13.5 

18 

10 

7.5 

a Average 

: Performance 

Index 

ranked 

1 to 

18  for 

the 

more  than 

six  months  group. 
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TABLE  C-3 — Continued 


API  and 

Personality 

Traits 

Subject  Number 

46 

47 

48 

49 

50 

51 

52 

53 

54 

APia 

7 

13 

12 

10 

4 

14 

11 

1 

3 

Cautiousness 

4.5 

6 

1 

17.5 

2.5 

14.5 

4.5 

9 

2.5 

Ascendancy 

7 

10 

10 

1 

8 

2 

4 

5.5 

10 

Vigor 

13.5 

13.5 

10.  5 

5.5 

15 

1 

10.5 

3 

4 

Personal 

Relations 

6.5 

5 

1 

4 

15 

9.5 

3 

16.5 

6.5 

Sociability 

3 

8 

5 

2 

5 

1 

5 

11 

8 

Original 

Thinking 

13.5 

5 

8 

6 

16 

4 

8 

2 

11 

Emotional 

Stability 

7 

2.5 

1 

13 

5.5 

17 

4 

5.5 

10 

Responsibility 

9 

7.5 

1.5 

13.5 

11.5 

15.5 

6 

4.5 

1.5 

a Aver age 

Performance 

Index 

ranked 

1 to 

18  for 

the 

more-than- 

six-months  group. 
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TABLE  C-4 


TABLE  C-4 — Continued 


jj  t* 


Significant  at  a = .05. 


TABLE  C-5 


SPEARMAN  RANK  ORDERING— FPTA/LTTA  AND  PERSONALITY  TEST  SCORES 


Subject  Number 


FPTA/LTTA  and 

Personality 

Traits 


Cautiousness 

Ascendancy 

Vigor 

Personal 

Relations 


TABLE  C-5 — Continued 


r 


FPTA/LTTA  and 

Personality 

Traits 

Subject  Number 

10* 

11 

12 

13 

14 

15 

16 

17 

18 

FPTA 

- 

10 

17 

23 

51 

33 

28 

24 

41 

LTTA 

- 

44 

25 

29 

38 

52 

48 

46 

39 

Cautiousness 

- 

35.5 

46 

35.5 

18.5 

24 

18.5 

44.5 

29.5 

Ascendancy 

- 

21.5 

4.5 

16.5 

29.  5 

34.5 

49.5 

29.5 

16.5 

Vigor 

- 

10 

7 

31 

35 

1 

50.5 

38 

26.5 

Personal 

Relations 

- 

41 

36 

34 

8.5 

36 

10.5 

8.5 

13 

Sociability 

- 

27.5 

36.5 

27.5 

12 

36.5 

15.5 

15.5 

4.5 

Original 

Thinking 

- 

35.5 

15 

15 

2.5 

18 

39.5 

17 

43 

Emotional 

Stability 

- 

46 

18.5 

40.5 

29 

9 

46 

40.5 

23 

Responsibility 

- 

43.5 

46.5 

43.5 

7.5 

7.5 

41 

30 

22 

TABLE  C-5 — Continued 


FPTA/LTTA  and 

Personality 

Traits 

Sub j ect 

Number 

19 

20 

21 

22 

23 

24 

25 

26 

27 

FPTA 

50 

9 

11 

40 

15 

22 

26 

7 

12 

LTTA 

31 

37 

2 

22 

1 

7 

10 

41 

24 

Cautiousness 

40 

40 

6 

27.5 

13 

31 

13 

48 

44.5 

Ascendancy 

10.5 

29.5 

10.5 

10.5 

47 

29.5 

40 

38.5 

10.5 

Vigor 

18.5 

23 

2.5 

7 

49 

47.5 

28 

44.5 

31 

Personal 

Relations 

19.5 

24 

41 

15 

16 

1.5 

41 

41 

3.5 

Sociability 

27.5 

33 

38.5 

7 

50.5 

21 

52 

33 

2 

Original 

Thinking 

30.5 

26.5 

19.5 

5 

43 

48.5 

35.5 

43 

11.5 

Bnotional 

Stability 

16.5 

33 

23 

29 

29 

14 

23 

49 

46 

Responsibility 

11 

38 

11 

3 

35 

30 

38 

50 

38 

r 
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TABLE  C-5 — Continued 


TABLE  C-5 — Continued 


FPTA/LTTA  and 

Personality 

Traits 

Subject 

Number 

46 

47 

48 

49 

50 

51 

52 

53 

54 

FPTA 

30 

35 

18 

46 

27 

45 

16 

1 

6 

LTTA 

26 

17 

11 

35 

5 

43 

2 

13 

23 

Cautiousness 

6 

9.5 

12 

50.5 

3 

40 

6 

18.5 

3 

Ascendancy 

21.5 

34.5 

34.5 

1 

29.5 

2 

10.5 

16.5 

34.5 

Vigor 

41 

41 

31 

15 

44.5 

2.5 

31 

7 

11.5 

Personal 

Relations 

19.5 

13 

1.5 

6.5 

45.5 

28 

5 

48 

19.5 

Sociability 

15.5 

27.5 

21 

3 

21 

1 

21 

38.5 

27.5 

Original 

Thinking 

35.5 

19.5 

26.5 

21.5 

50 

15 

26.5 

8 

30.5 

EJnotional 

Stability 

16.5 

2.5 

1 

40.5 

9 

52 

4.5 

9 

29 

Responsibility 

22 

18 

1.5 

41 

35 

46.5 

15 

11 

1.5 

TABLE  06 


aSpearman  Rank  Order  Correlation  FPTA  Performance  Index  ranked  1 to  53  with  Gordon 
Personality  Trait  Scores  ranked  1 to  53. 


TABLE  C-6 — Continued 


.012) 


TABLE  C- 


aSpearman  Rank  Order  Correlation  LTTA  Performance  Index  ranked  1 to  53  with  Gordon 
Personality  Trait  Scores  ranked  1 to  53. 


TABLE  C-7 — Continued 


(12,402) (12,335) 


TABLE  C-8 


SPEARMAN  RANK  ORDERING — T.O.  AND  GORDON  PERSONALITY  TEST  SCORES 


T.O.  and 

Personality 

Traits 

Subject  Number 

19 

20 

21 

22 

23 

24 

25 

26 

27 

T.O. 

27 

33 

8 

22 

9 

14 

6.5 

30 

32 

Cautiousness 

29 

29 

6 

22 

13 

23 

13 

33.5 

32 

Ascendancy 

9.5 

19.5 

9.5 

9.5 

31.5 

19.5 

26 

25 

9.5 

Vigor 

13 

16.5 

1.5 

5.5 

34 

32.5 

20 

30 

22.5 

Personal 

Relations 

12.5 

17 

28 

8 

9 

1.5 

28 

28 

3.5 

Sociability 

17.5 

21.5 

24.5 

4.5 

34 

12 

35 

21.  5 

2 

Original 

Thinking 

21.5 

17.5 

11.  5 

3 

29.5 

32.5 

25 

29.5 

8 

Emotional 

Stability 

12.5 

24 

16.5 

21 

21 

10.5 

16.5 

33 

31 

Responsibility 

8 

27 

8 

3 

24 

20.5 

27 

35 

27 

TABLE  C-8 — Continued 


T.O.  and 

Personality 

Traits 



Subject  Number 

28 

29 

30 

31 

32 

33 

34 

35 

36 

T.O. 

23 

20 

18 

6.5 

34 

36 

29 

21 

17 

Cautiousness 

17 

13 

9.5 

8 

29 

25.5 

25.5 

17 

3 

Ascendancy 

19.5 

9.5 

3 

16 

5.5 

5.5 

31.5 

17 

34 

Vigor 

27 

13 

5.5 

16.5 

5.5 

16.5 

30 

22.5 

8.5 

Personal 

Relations 

12.5 

31.5 

20 

17 

12.5 

12.5 

35.5 

22.5 

25 

Sociability 

12 

4.5 

7 

12 

6 

12 

36 

21.5 

29.5 

Original 

Thinking 

32.5 

27 

13.5 

1 

5 

8 

29.5 

5 

15 

Emotional 

Stability 

16.5 

16.5 

10.5 

7.5 

26.5 

7.5 

24 

4.5 

28.  5 

Responsibility 

20.5 

20.5 

14 

11.5 

20.5 

8 

32 

4.5 

16.5 

TABLE  C-8 — Continued 


T.O.  and 

Personality 

Traits 

Subject  Number 

37 

38 

39 

40 

41 

42 

43 

44 

45 

T.O. 

31 

12 

4 

3 

25 

15 

35 

1 

10 

Cautiousness 

20 

35.5 

20 

20 

29 

13 

13 

33.5 

24 

Ascendancy 

35 

27 

29.5 

13.5 

28 

33 

29.5 

36 

4 

Vigor 

16.5 

10.5 

25 

19 

13 

32.5 

36 

35 

3 

Personal 

Relations 

20 

28 

24 

3.5 

22.5 

28 

17 

35.5 

33.5 

Soc iability 

26.5 

29.5 

32.5 

21.5 

29.5 

26.5 

29.5 

32.5 

17.5 

Original 

Thinking 

29.5 

17.5 

25 

8 

21.5 

35.5 

21.5 

35.5 

2 

Emotional 

Stability 

30 

36 

24 

2.5 

32 

16.5 

26.5 

34 

16.5 

Responsibil ity 

24 

32 

8 

4.5 

34 

29.5 

36 

18 

14 
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TABLE  08 — Continued 


T.O.  and  Subject  Number 

Personality  


Traits 

46 

47 

48 

49 

50 

51 

52 

53 

54 

T.O. 

11 

19 

26 

24 

13 

16 

28 

2 

5 

Cautiousness 

26 

6 

9.5 

1 

35.5 

3 

29 

6 

17 

Ascendancy 

15 

23 

23 

1 

19.5 

2 

9.5 

13.5 

23 

Vigor 

27 

27 

22.5 

10.5 

30 

1.5 

22.5 

5.5 

8.5 

Personal 

Relations 

12.5 

7 

1.5 

6 

31.5 

20 

5 

33.5 

12.5 

Sociability 

8 

17.5 

12 

3 

12 

1 

12 

24.5 

17.5 

Original 

Thinking 

25 

11.5 

17.5 

13.5 

34 

10 

17.5 

5 

21.5 

Emotional 

Stability 

12.5 

2.5 

1 

28.5 

7.5 

35 

4.5 

7.5 

21 

Respons ibil ity 

16.5 

14 

1.5 

29.5 

24 

32 

11.5 

8 

1.5 

TABLE  C- 


aSpeannan  Rank  Order  Correlation  T.O.  Performance  Index  ranked  1 to  36  with  Gordon 
Personality  Traits  ranked  1 to  36. 


(3,884.5) (3,859.5) 


APPENDIX  D 

SPEARMAN  RANK  CORRELATION  COEFFICIENT:  AQE  SCORES 
WITH  PERFORMANCE  INDICES 


TABLE  D-l 


AQE  SCORES 


Subject  Number 


Aptitude 

Clusters 


8 


General 

85 

95 

85 

95 

95 

85 

95 

85 

85 

Administrative 

30 

55 

55 

55 

85 

60 

80 

70 

70 

Mechanical 

65 

80 

95 

55 

35 

90 

95 

85 

60 

Electronics 

85 

95 

95 

95 

90 

85 

95 

85 

95 

♦Subject  47  did  not  have  AQE  aptitude  cluster  scores. 
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TABLE  D-2 


SPEARMAN  RANK  ORDERING— API  AND  AQE  SCORES— 

INTERMEDIATE  LEVEL  MAINTENANCE:  MORE- THAN- SIX-MONTHS -EXPERIENCE  GROUP 


API  and 

Aptitude 

Rankings 

Subject  Number 

38 

39 

40 

41 

42 

43 

API* 

16 

6 

8 

5 

14 

15 

17 

2 

9 

General 

13 

17 

13 

7.5 

2 

9.5 

9.5 

16 

2 

Administrative 

15 

16.5 

10 

1.5 

4.5 

12.5 

10 

16.  5 

4.5 

Mechanical 

5 

14 

14 

17 

12 

10.5 

5 

14 

5 

Electronics 

15.5 

12.5 

3.5 

8.5 

15.5 

8.5 

3.5 

12.5 

3.5 

TABLE  D-3 


(408) (380) 


TABLE  D-4 


SPEARMAN  RANK  ORDERING — FPTA/LTTA  AND  AQE  SCORES 


FPTA,  LTTA 
Ranking  of 
Aptitude 
Clusters 

Sub j ect 

Number 

1 

2 

3 

4 

5 

6 

7 

8 

9 

FPTA 

34 

46 

35 

19 

52 

48 

13 

14 

37 

LTTA 

51 

49 

19 

31 

50 

35 

32 

44 

27 

General 

22 

5.5 

22 

5.5 

5.5 

22 

5.5 

22 

22 

Administrative 

52 

43 

43 

43 

10 

37 

16.5 

27 

27 

Mechanical 

42.5 

29 

3.5 

49.5 

53 

10.5 

3.5 

20.5 

46.5 

Electronics 

36.5 

9 

9 

9 

15.5 

36.5 

9 

36.5 

9 

FPTA,  LTTA 
Ranking  of 
Aptitude 
Clusters 

Subject  Number 

10 

11 

12 

13 

14 

15 

16 

17 

18 

FPTA 

50 

10 

17 

23 

51 

33 

28 

24 

41 

LTTA 

45 

43 

24 

28 

37 

52 

48 

46 

38 

General 

38.5 

5.5 

49.5 

38.5 

38.5 

32.5 

52 

5.5 

44.5 

Administrative 

32.5 

10 

37 

53 

37 

16.5 

47 

37 

47 

Mechanical 

51 

52 

33 

20.5 

20.5 

29 

10.5 

3.5 

42.5 

Electronics 

36.5 

9 

9 

36.5 

15.5 

47.5 

15.5 

9 

47.5 
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Subject  Number 


FPTA,  LTTA 
Ranking  of 
Aptitude 


Clusters 

19 

20 

21 

22 

23 

24 

25 

26 

27 

FPTA 

49 

9 

11 

39 

15 

22 

26 

1 

12 

LTTA 

30 

36 

3 

21 

1 

7 

10 

40 

23 

General 

44.5 

22 

32.5 

22 

32.5 

13 

22 

22 

13 

Administrative 

21.5 

10 

16.5 

27 

16.5 

43 

10 

27 

37 

Mechanical 

29 

20.5 

20.  5 

3.5 

33 

46.5 

20.5 

20.5 

46.5 

Electronics 

47.5 

15.5 

15.5 

9 

47.5 

15.5 

36.5 

36.5 

47.5 

FPTA,  LTTA 
Ranking  of 
Aptitude 
Clusters 


Subject  Number 


28  29  30  31  32  33  34  35  36 


TABLE  D-4 — Continued 


FPTA,  LTTA 
Banking  of 
Aptitude 
Clusters 

Subject  Number 

• 

38 

39 

40 

41 

42 

43 

FPTA 

5 

m 

36 

30 

38 

53 

2 

43 

I 

LTTA 

14 

29 

39 

53 

33 

12 

General 

44.5 

53 

44.5 

22 

5.5 

32.5 

32.5 

49.5 

5.5 

Administrative 

47 

50 

27 

2 

5.5 

37 

27 

50 

5.5 

Mechanical 

10.5 

37.5 

37.5 

49.5 

33 

29 

10.5 

37.5 

10.5 

Electronics 

47.5 

36.5 

9 

15.5 

47.5 

15.5 

9 

36.5 

9 

FPTA,  LTTA 
Banking  of 
Aptitude 
Clusters 

Subject 

Number 

46 

49 

51 

52 

53 

54 

FPTA 

30 

- 

18 

45 

27 

44 

16 

1 

6 

LTTA 

25 

- 

11 

34 

5 

42 

2 

13 

22 

General 

13 

- 

13 

44.5 

22 

5.5 

44.5 

13 

44.5 

Administrative 

21.5 

- 

2 

16.5 

5.5 

5.5 

27 

37 

43 

Mechanical 

3.5 

- 

3.5 

29 

20.5 

10.5 

42.5 

10.5 

20.5 

Electronics 

9 

- 

15.5 

36.5 

15.5 

9 

53 

9 

36.5 

•Subject  47  did  not  have  AQE  aptitude  cluster  scores. 


(12,402) (11,563.5) 


TABLE  D-7 

SPEARMAN  RANK  ORDERING — T.O.  AND  AQE  TEST  SCORES 


T.O. 

Ranking  of 

Aptitude 

Clusters 

. a 

Subject  Number 

19 

20 

21 

22 

23 

24 

25 

26 

27 

T.0.b 

26 

32 

8 

21 

8 

14 

6.5 

29 

31 

General 

28.5 

13.5 

21 

13.5 

21 

7 

13.5 

13.5 

7 

Administrative 

17.5 

9 

13.5 

22 

13.5 

30.5 

9 

22 

28 

Mechanical 

20 

14 

14 

2 

22.5 

32 

14 

14 

32 

Electronics 

31 

14 

14 

5 

31 

14 

23 

23 

31 

T.O. 

Ranking  of 

Aptitude 

Clusters 

. a 

Subject  Number 

28 

29 

30 

31 

32 

33 

34 

35 

36 

T.O.b 

22 

19 

18 

6.5 

33 

35 

28 

20 

17 

General 

21 

34 

13.5 

2.5 

13.5 

21 

28.5 

21 

25 

Administrative 

22 

34 

13.5 

9 

26 

22 

2 

13.5 

13.5 

Mechanical 

14 

26 

14 

6.5 

27.5 

32 

26 

26 

14 

Electronics 

14 

23 

31 

5 

14 

23 

23 

31 

5 

aZero  experience  group  was  not  tested  with  T.O. 
bT.O.  performance  index  ranked  1 to  35. 
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TABLE  D-7 — Continued 


Ranking  of 

Aptitude 

Clusters 


General 

Administrate 

Mechanical 

Electronics 


. a 

Subject  Number 

37 

38 

39 

40 

41 

42 

43 

44 

45 

30 

11 

4 

3 

24 

15 

34 

1 

10 

• 1 

28.5 

35 

28.5 

13.5 

2.5 

21 

21 

33 

2.5 

32 

34 

22 

2 

4.5 

28 

22 

34 

4.5 

6.5 

26 

26 

34 

22.5 

20 

6.5 

26 

6.5 

31 

23 

5 

14 

31 

14 

5 

27 

5 

Ranking  of 

Aptitude 

Clusters 


General 

Administrative  ) 17 . 5 
Mechanical 
Electronics 


Subject 

Number 

a 

46 

47* 

48 

49 

50 

51 

52 

53 

54 

11 

- 

25 

23 

13 

16 

27 

2 

5 

7 

- 

7 

28.5 

13.5 

2.5 

28.5 

7 

28.5 

17.5 

- 

2 

13.5 

4.5 

4.5 

22 

28 

30.5 

2 

- 

2 

20 

14 

6.5 

27.5 

6.5 

14 

5 

- 

14 

27 

14 

5 

35 

5 

23 

• 

Zero  experience  group  was  not  tested  with  T.O. 

“t.O.  performance  index  ranked  1 to  35. 

•Subject  47  did  not  have  AQE  aptitude  cluster  scores 
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SPEARMAN  RANK  CORRELATION— T. O.  WTTH  AQE  SCORES 


r 


(3569.5) (3362) 
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